Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_no_mpnn_from_auc/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_no_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 5,
 'device': device(type='cuda', index=1),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': True,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 800,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_no_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.531455
Epoch 1
Validation auc = 0.601408
Epoch 2
Validation auc = 0.623944
Epoch 3
Validation auc = 0.672770
Epoch 4
Loss = 7.2248e-01, PNorm = 20.9844, GNorm = 7.5491, lr_0 = 6.5000e-04
Validation auc = 0.684507
Epoch 5
Validation auc = 0.658685
Epoch 6
Validation auc = 0.611268
Epoch 7
Validation auc = 0.601408
Epoch 8
Validation auc = 0.610798
Epoch 9
Loss = 5.5054e-01, PNorm = 21.0311, GNorm = 2.0930, lr_0 = 9.9743e-04
Validation auc = 0.605634
Epoch 10
Validation auc = 0.657746
Epoch 11
Validation auc = 0.680282
Epoch 12
Validation auc = 0.704225
Epoch 13
Validation auc = 0.705634
Epoch 14
Loss = 3.8720e-01, PNorm = 21.0932, GNorm = 4.6824, lr_0 = 9.8890e-04
Validation auc = 0.709390
Epoch 15
Validation auc = 0.712207
Epoch 16
Validation auc = 0.716901
Epoch 17
Validation auc = 0.714554
Epoch 18
Validation auc = 0.701408
Epoch 19
Loss = 4.7277e-01, PNorm = 21.1568, GNorm = 2.4497, lr_0 = 9.8045e-04
Validation auc = 0.717840
Epoch 20
Validation auc = 0.716432
Epoch 21
Validation auc = 0.686854
Epoch 22
Validation auc = 0.682629
Epoch 23
Validation auc = 0.708451
Epoch 24
Loss = 3.9325e-01, PNorm = 21.2281, GNorm = 2.1612, lr_0 = 9.7207e-04
Validation auc = 0.718310
Epoch 25
Validation auc = 0.728638
Epoch 26
Validation auc = 0.726291
Epoch 27
Validation auc = 0.730516
Epoch 28
Validation auc = 0.726761
Epoch 29
Loss = 3.1398e-01, PNorm = 21.2973, GNorm = 3.1595, lr_0 = 9.6376e-04
Validation auc = 0.716432
Epoch 30
Validation auc = 0.713146
Epoch 31
Validation auc = 0.713146
Epoch 32
Validation auc = 0.714554
Epoch 33
Validation auc = 0.723005
Epoch 34
Loss = 3.0659e-01, PNorm = 21.3814, GNorm = 5.7509, lr_0 = 9.5552e-04
Validation auc = 0.719249
Epoch 35
Validation auc = 0.706103
Epoch 36
Validation auc = 0.700939
Epoch 37
Validation auc = 0.710798
Epoch 38
Validation auc = 0.721127
Epoch 39
Loss = 3.9384e-01, PNorm = 21.4551, GNorm = 2.3863, lr_0 = 9.4735e-04
Validation auc = 0.717840
Epoch 40
Validation auc = 0.718310
Epoch 41
Validation auc = 0.718310
Epoch 42
Validation auc = 0.723005
Epoch 43
Validation auc = 0.711737
Epoch 44
Loss = 3.8111e-01, PNorm = 21.5357, GNorm = 2.3602, lr_0 = 9.3925e-04
Validation auc = 0.715962
Epoch 45
Validation auc = 0.713615
Epoch 46
Validation auc = 0.716432
Epoch 47
Validation auc = 0.710798
Epoch 48
Validation auc = 0.704225
Epoch 49
Loss = 3.2933e-01, PNorm = 21.6107, GNorm = 3.1026, lr_0 = 9.3122e-04
Validation auc = 0.705164
Epoch 50
Validation auc = 0.713615
Epoch 51
Validation auc = 0.713146
Epoch 52
Validation auc = 0.711268
Epoch 53
Validation auc = 0.708451
Epoch 54
Loss = 3.4921e-01, PNorm = 21.6930, GNorm = 2.3100, lr_0 = 9.2326e-04
Validation auc = 0.702817
Epoch 55
Validation auc = 0.706103
Epoch 56
Validation auc = 0.702347
Epoch 57
Validation auc = 0.698592
Epoch 58
Validation auc = 0.700469
Epoch 59
Loss = 2.9752e-01, PNorm = 21.7747, GNorm = 2.0968, lr_0 = 9.1537e-04
Validation auc = 0.696714
Epoch 60
Validation auc = 0.702817
Epoch 61
Validation auc = 0.707981
Epoch 62
Validation auc = 0.712676
Epoch 63
Validation auc = 0.717840
Epoch 64
Loss = 2.8317e-01, PNorm = 21.8693, GNorm = 5.7727, lr_0 = 9.0754e-04
Validation auc = 0.720657
Epoch 65
Validation auc = 0.723005
Epoch 66
Validation auc = 0.721596
Epoch 67
Validation auc = 0.713615
Epoch 68
Validation auc = 0.703756
Epoch 69
Loss = 3.3261e-01, PNorm = 21.9605, GNorm = 1.5437, lr_0 = 8.9978e-04
Validation auc = 0.714085
Epoch 70
Validation auc = 0.715023
Epoch 71
Validation auc = 0.706103
Epoch 72
Validation auc = 0.697653
Epoch 73
Validation auc = 0.687324
Epoch 74
Loss = 2.8798e-01, PNorm = 22.0538, GNorm = 2.6078, lr_0 = 8.9209e-04
Validation auc = 0.682160
Epoch 75
Validation auc = 0.686385
Epoch 76
Validation auc = 0.689671
Epoch 77
Validation auc = 0.692958
Epoch 78
Validation auc = 0.699061
Epoch 79
Loss = 2.7824e-01, PNorm = 22.1437, GNorm = 2.1155, lr_0 = 8.8447e-04
Validation auc = 0.693897
Epoch 80
Validation auc = 0.697653
Epoch 81
Validation auc = 0.698592
Epoch 82
Validation auc = 0.696244
Epoch 83
Validation auc = 0.695775
Epoch 84
Loss = 3.0692e-01, PNorm = 22.2335, GNorm = 6.2582, lr_0 = 8.7691e-04
Validation auc = 0.695305
Epoch 85
Validation auc = 0.696714
Epoch 86
Validation auc = 0.673709
Epoch 87
Validation auc = 0.702347
Epoch 88
Validation auc = 0.707042
Epoch 89
Loss = 2.6255e-01, PNorm = 22.3201, GNorm = 1.6541, lr_0 = 8.6941e-04
Validation auc = 0.701878
Epoch 90
Validation auc = 0.701408
Epoch 91
Validation auc = 0.702347
Epoch 92
Validation auc = 0.697653
Epoch 93
Validation auc = 0.693897
Epoch 94
Loss = 2.3465e-01, PNorm = 22.4014, GNorm = 1.6915, lr_0 = 8.6198e-04
Validation auc = 0.684507
Epoch 95
Validation auc = 0.689202
Epoch 96
Validation auc = 0.686854
Epoch 97
Validation auc = 0.690610
Epoch 98
Validation auc = 0.689202
Epoch 99
Loss = 2.9778e-01, PNorm = 22.4831, GNorm = 1.5981, lr_0 = 8.5461e-04
Validation auc = 0.688732
Epoch 100
Validation auc = 0.691080
Epoch 101
Validation auc = 0.677934
Epoch 102
Validation auc = 0.678404
Epoch 103
Validation auc = 0.659624
Epoch 104
Loss = 2.6225e-01, PNorm = 22.5696, GNorm = 2.4028, lr_0 = 8.4730e-04
Validation auc = 0.631455
Epoch 105
Validation auc = 0.628638
Epoch 106
Validation auc = 0.631455
Epoch 107
Validation auc = 0.661502
Epoch 108
Validation auc = 0.674178
Epoch 109
Loss = 2.5791e-01, PNorm = 22.6488, GNorm = 1.3674, lr_0 = 8.4006e-04
Validation auc = 0.685446
Epoch 110
Validation auc = 0.686385
Epoch 111
Validation auc = 0.690141
Epoch 112
Validation auc = 0.689671
Epoch 113
Validation auc = 0.680751
Epoch 114
Loss = 3.2707e-01, PNorm = 22.7289, GNorm = 3.3042, lr_0 = 8.3288e-04
Validation auc = 0.680282
Epoch 115
Validation auc = 0.673709
Epoch 116
Validation auc = 0.664319
Epoch 117
Validation auc = 0.676056
Epoch 118
Validation auc = 0.683568
Epoch 119
Loss = 2.0998e-01, PNorm = 22.8054, GNorm = 2.2681, lr_0 = 8.2576e-04
Validation auc = 0.683099
Epoch 120
Validation auc = 0.687793
Epoch 121
Validation auc = 0.689671
Epoch 122
Validation auc = 0.688263
Epoch 123
Validation auc = 0.684038
Epoch 124
Loss = 2.4927e-01, PNorm = 22.8802, GNorm = 1.8821, lr_0 = 8.1870e-04
Validation auc = 0.684507
Epoch 125
Validation auc = 0.677934
Epoch 126
Validation auc = 0.674648
Epoch 127
Validation auc = 0.671362
Epoch 128
Validation auc = 0.670423
Epoch 129
Loss = 1.5595e-01, PNorm = 22.9497, GNorm = 1.4096, lr_0 = 8.1170e-04
Validation auc = 0.677934
Epoch 130
Validation auc = 0.689202
Epoch 131
Validation auc = 0.693427
Epoch 132
Validation auc = 0.701408
Epoch 133
Validation auc = 0.700939
Epoch 134
Loss = 2.5752e-01, PNorm = 23.0167, GNorm = 1.6502, lr_0 = 8.0476e-04
Validation auc = 0.698122
Epoch 135
Validation auc = 0.691549
Epoch 136
Validation auc = 0.687793
Epoch 137
Validation auc = 0.677934
Epoch 138
Validation auc = 0.675117
Epoch 139
Loss = 2.5885e-01, PNorm = 23.0916, GNorm = 2.1968, lr_0 = 7.9788e-04
Validation auc = 0.683568
Epoch 140
Validation auc = 0.684977
Epoch 141
Validation auc = 0.686854
Epoch 142
Validation auc = 0.679812
Epoch 143
Validation auc = 0.670423
Epoch 144
Loss = 1.9781e-01, PNorm = 23.1615, GNorm = 1.0939, lr_0 = 7.9106e-04
Validation auc = 0.671831
Epoch 145
Validation auc = 0.679343
Epoch 146
Validation auc = 0.684507
Epoch 147
Validation auc = 0.685446
Epoch 148
Validation auc = 0.690610
Epoch 149
Loss = 2.2468e-01, PNorm = 23.2330, GNorm = 2.8057, lr_0 = 7.8430e-04
Validation auc = 0.689671
Epoch 150
Validation auc = 0.692019
Epoch 151
Validation auc = 0.690610
Epoch 152
Validation auc = 0.691080
Epoch 153
Validation auc = 0.689202
Epoch 154
Loss = 2.3584e-01, PNorm = 23.3124, GNorm = 4.3529, lr_0 = 7.7759e-04
Validation auc = 0.688263
Epoch 155
Validation auc = 0.692958
Epoch 156
Validation auc = 0.690141
Epoch 157
Validation auc = 0.699531
Epoch 158
Validation auc = 0.703756
Epoch 159
Loss = 1.6298e-01, PNorm = 23.3850, GNorm = 4.0388, lr_0 = 7.7095e-04
Validation auc = 0.704695
Epoch 160
Validation auc = 0.700000
Epoch 161
Validation auc = 0.705634
Epoch 162
Validation auc = 0.698592
Epoch 163
Validation auc = 0.695305
Epoch 164
Loss = 2.8895e-01, PNorm = 23.4572, GNorm = 1.6907, lr_0 = 7.6436e-04
Validation auc = 0.684977
Epoch 165
Validation auc = 0.666197
Epoch 166
Validation auc = 0.671831
Epoch 167
Validation auc = 0.696714
Epoch 168
Validation auc = 0.699061
Epoch 169
Loss = 1.6048e-01, PNorm = 23.5275, GNorm = 3.0259, lr_0 = 7.5782e-04
Validation auc = 0.701878
Epoch 170
Validation auc = 0.703286
Epoch 171
Validation auc = 0.699531
Epoch 172
Validation auc = 0.702347
Epoch 173
Validation auc = 0.694366
Epoch 174
Loss = 2.3290e-01, PNorm = 23.5905, GNorm = 0.8672, lr_0 = 7.5134e-04
Validation auc = 0.685915
Epoch 175
Validation auc = 0.675587
Epoch 176
Validation auc = 0.664789
Epoch 177
Validation auc = 0.670423
Epoch 178
Validation auc = 0.672300
Epoch 179
Loss = 2.2512e-01, PNorm = 23.6493, GNorm = 1.6285, lr_0 = 7.4492e-04
Validation auc = 0.684507
Epoch 180
Validation auc = 0.693897
Epoch 181
Validation auc = 0.694366
Epoch 182
Validation auc = 0.698592
Epoch 183
Validation auc = 0.697653
Epoch 184
Loss = 1.5966e-01, PNorm = 23.7190, GNorm = 1.7784, lr_0 = 7.3855e-04
Validation auc = 0.701878
Epoch 185
Validation auc = 0.706573
Epoch 186
Validation auc = 0.710798
Epoch 187
Validation auc = 0.713615
Epoch 188
Validation auc = 0.710329
Epoch 189
Loss = 1.8544e-01, PNorm = 23.7857, GNorm = 1.5749, lr_0 = 7.3224e-04
Validation auc = 0.706103
Epoch 190
Validation auc = 0.700939
Epoch 191
Validation auc = 0.693427
Epoch 192
Validation auc = 0.684507
Epoch 193
Validation auc = 0.685446
Epoch 194
Loss = 2.0582e-01, PNorm = 23.8408, GNorm = 2.5581, lr_0 = 7.2598e-04
Validation auc = 0.679812
Epoch 195
Validation auc = 0.675117
Epoch 196
Validation auc = 0.668545
Epoch 197
Validation auc = 0.663380
Epoch 198
Validation auc = 0.663380
Epoch 199
Loss = 2.0170e-01, PNorm = 23.8990, GNorm = 2.6202, lr_0 = 7.1977e-04
Validation auc = 0.666197
Epoch 200
Validation auc = 0.667606
Epoch 201
Validation auc = 0.669014
Epoch 202
Validation auc = 0.675587
Epoch 203
Validation auc = 0.686385
Epoch 204
Loss = 1.5884e-01, PNorm = 23.9583, GNorm = 2.4583, lr_0 = 7.1362e-04
Validation auc = 0.687324
Epoch 205
Validation auc = 0.692019
Epoch 206
Validation auc = 0.692488
Epoch 207
Validation auc = 0.693427
Epoch 208
Validation auc = 0.692958
Epoch 209
Loss = 2.0482e-01, PNorm = 24.0248, GNorm = 2.2089, lr_0 = 7.0752e-04
Validation auc = 0.686385
Epoch 210
Validation auc = 0.685915
Epoch 211
Validation auc = 0.681690
Epoch 212
Validation auc = 0.680282
Epoch 213
Validation auc = 0.677465
Epoch 214
Loss = 2.2756e-01, PNorm = 24.0843, GNorm = 2.2324, lr_0 = 7.0147e-04
Validation auc = 0.678404
Epoch 215
Validation auc = 0.680282
Epoch 216
Validation auc = 0.684507
Epoch 217
Validation auc = 0.686385
Epoch 218
Validation auc = 0.686385
Epoch 219
Loss = 1.5835e-01, PNorm = 24.1452, GNorm = 2.4295, lr_0 = 6.9548e-04
Validation auc = 0.684038
Epoch 220
Validation auc = 0.683099
Epoch 221
Validation auc = 0.679343
Epoch 222
Validation auc = 0.680282
Epoch 223
Validation auc = 0.673709
Epoch 224
Loss = 1.7237e-01, PNorm = 24.2078, GNorm = 3.8143, lr_0 = 6.8953e-04
Validation auc = 0.674178
Epoch 225
Validation auc = 0.677934
Epoch 226
Validation auc = 0.676056
Epoch 227
Validation auc = 0.679343
Epoch 228
Validation auc = 0.686385
Epoch 229
Loss = 2.1918e-01, PNorm = 24.2628, GNorm = 4.3187, lr_0 = 6.8364e-04
Validation auc = 0.680282
Epoch 230
Validation auc = 0.676995
Epoch 231
Validation auc = 0.678873
Epoch 232
Validation auc = 0.673709
Epoch 233
Validation auc = 0.662911
Epoch 234
Loss = 1.3570e-01, PNorm = 24.3334, GNorm = 1.3273, lr_0 = 6.7779e-04
Validation auc = 0.656808
Epoch 235
Validation auc = 0.661502
Epoch 236
Validation auc = 0.662441
Epoch 237
Validation auc = 0.659624
Epoch 238
Validation auc = 0.661502
Epoch 239
Loss = 1.5201e-01, PNorm = 24.3930, GNorm = 2.2692, lr_0 = 6.7200e-04
Validation auc = 0.660094
Epoch 240
Validation auc = 0.661972
Epoch 241
Validation auc = 0.661502
Epoch 242
Validation auc = 0.661033
Epoch 243
Validation auc = 0.661972
Epoch 244
Loss = 2.1097e-01, PNorm = 24.4521, GNorm = 1.7334, lr_0 = 6.6625e-04
Validation auc = 0.663850
Epoch 245
Validation auc = 0.667136
Epoch 246
Validation auc = 0.666197
Epoch 247
Validation auc = 0.667606
Epoch 248
Validation auc = 0.668075
Epoch 249
Loss = 2.3584e-01, PNorm = 24.5096, GNorm = 5.7734, lr_0 = 6.6056e-04
Validation auc = 0.670423
Epoch 250
Validation auc = 0.671831
Epoch 251
Validation auc = 0.673239
Epoch 252
Validation auc = 0.668545
Epoch 253
Validation auc = 0.667606
Epoch 254
Loss = 1.6108e-01, PNorm = 24.5676, GNorm = 1.4678, lr_0 = 6.5491e-04
Validation auc = 0.672770
Epoch 255
Validation auc = 0.671831
Epoch 256
Validation auc = 0.669484
Epoch 257
Validation auc = 0.660094
Epoch 258
Validation auc = 0.655399
Epoch 259
Loss = 1.4594e-01, PNorm = 24.6258, GNorm = 0.9108, lr_0 = 6.4931e-04
Validation auc = 0.655869
Epoch 260
Validation auc = 0.665728
Epoch 261
Validation auc = 0.668545
Epoch 262
Validation auc = 0.668545
Epoch 263
Validation auc = 0.672300
Epoch 264
Loss = 3.0699e-01, PNorm = 24.6799, GNorm = 5.1012, lr_0 = 6.4376e-04
Validation auc = 0.677465
Epoch 265
Validation auc = 0.678404
Epoch 266
Validation auc = 0.681221
Epoch 267
Validation auc = 0.678873
Epoch 268
Validation auc = 0.675117
Epoch 269
Loss = 1.3837e-01, PNorm = 24.7399, GNorm = 1.0123, lr_0 = 6.3826e-04
Validation auc = 0.672300
Epoch 270
Validation auc = 0.668545
Epoch 271
Validation auc = 0.669484
Epoch 272
Validation auc = 0.669484
Epoch 273
Validation auc = 0.677934
Epoch 274
Loss = 1.5080e-01, PNorm = 24.7975, GNorm = 3.0860, lr_0 = 6.3280e-04
Validation auc = 0.678404
Epoch 275
Validation auc = 0.682629
Epoch 276
Validation auc = 0.680282
Epoch 277
Validation auc = 0.679812
Epoch 278
Validation auc = 0.680282
Epoch 279
Loss = 1.6126e-01, PNorm = 24.8535, GNorm = 3.1419, lr_0 = 6.2739e-04
Validation auc = 0.684038
Epoch 280
Validation auc = 0.684977
Epoch 281
Validation auc = 0.680751
Epoch 282
Validation auc = 0.685915
Epoch 283
Validation auc = 0.688732
Epoch 284
Loss = 1.9320e-01, PNorm = 24.9031, GNorm = 2.3131, lr_0 = 6.2203e-04
Validation auc = 0.687324
Epoch 285
Validation auc = 0.693897
Epoch 286
Validation auc = 0.693897
Epoch 287
Validation auc = 0.689202
Epoch 288
Validation auc = 0.683099
Epoch 289
Loss = 9.1047e-02, PNorm = 24.9540, GNorm = 2.0928, lr_0 = 6.1671e-04
Validation auc = 0.679812
Epoch 290
Validation auc = 0.676056
Epoch 291
Validation auc = 0.680751
Epoch 292
Validation auc = 0.684038
Epoch 293
Validation auc = 0.681690
Epoch 294
Loss = 1.5252e-01, PNorm = 25.0031, GNorm = 3.7285, lr_0 = 6.1144e-04
Validation auc = 0.678404
Epoch 295
Validation auc = 0.669953
Epoch 296
Validation auc = 0.669484
Epoch 297
Validation auc = 0.670892
Epoch 298
Validation auc = 0.673709
Epoch 299
Loss = 2.1711e-01, PNorm = 25.0493, GNorm = 1.7102, lr_0 = 6.0621e-04
Validation auc = 0.676995
Model 0 best validation auc = 0.730516 on epoch 27
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.871764
Ensemble test auc = 0.871764
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.508451
Epoch 1
Validation auc = 0.619249
Epoch 2
Validation auc = 0.638028
Epoch 3
Validation auc = 0.661033
Epoch 4
Loss = 7.0630e-01, PNorm = 20.9836, GNorm = 3.1815, lr_0 = 6.5000e-04
Validation auc = 0.664319
Epoch 5
Validation auc = 0.638498
Epoch 6
Validation auc = 0.624413
Epoch 7
Validation auc = 0.615023
Epoch 8
Validation auc = 0.625822
Epoch 9
Loss = 5.6041e-01, PNorm = 21.0300, GNorm = 4.6393, lr_0 = 9.9743e-04
Validation auc = 0.645070
Epoch 10
Validation auc = 0.693427
Epoch 11
Validation auc = 0.711737
Epoch 12
Validation auc = 0.715962
Epoch 13
Validation auc = 0.719249
Epoch 14
Loss = 4.6756e-01, PNorm = 21.0943, GNorm = 2.2201, lr_0 = 9.8890e-04
Validation auc = 0.729577
Epoch 15
Validation auc = 0.730047
Epoch 16
Validation auc = 0.731455
Epoch 17
Validation auc = 0.717840
Epoch 18
Validation auc = 0.719718
Epoch 19
Loss = 4.1908e-01, PNorm = 21.1560, GNorm = 4.2717, lr_0 = 9.8045e-04
Validation auc = 0.704695
Epoch 20
Validation auc = 0.694366
Epoch 21
Validation auc = 0.686854
Epoch 22
Validation auc = 0.684038
Epoch 23
Validation auc = 0.676526
Epoch 24
Loss = 4.4993e-01, PNorm = 21.2214, GNorm = 7.8259, lr_0 = 9.7207e-04
Validation auc = 0.679812
Epoch 25
Validation auc = 0.679343
Epoch 26
Validation auc = 0.678873
Epoch 27
Validation auc = 0.674648
Epoch 28
Validation auc = 0.676995
Epoch 29
Loss = 3.7572e-01, PNorm = 21.2866, GNorm = 4.8486, lr_0 = 9.6376e-04
Validation auc = 0.681221
Epoch 30
Validation auc = 0.681690
Epoch 31
Validation auc = 0.692488
Epoch 32
Validation auc = 0.700469
Epoch 33
Validation auc = 0.700939
Epoch 34
Loss = 3.3972e-01, PNorm = 21.3577, GNorm = 6.4566, lr_0 = 9.5552e-04
Validation auc = 0.700000
Epoch 35
Validation auc = 0.698592
Epoch 36
Validation auc = 0.695775
Epoch 37
Validation auc = 0.692019
Epoch 38
Validation auc = 0.695305
Epoch 39
Loss = 3.8138e-01, PNorm = 21.4338, GNorm = 3.2153, lr_0 = 9.4735e-04
Validation auc = 0.699531
Epoch 40
Validation auc = 0.692488
Epoch 41
Validation auc = 0.688263
Epoch 42
Validation auc = 0.683099
Epoch 43
Validation auc = 0.681690
Epoch 44
Loss = 3.6525e-01, PNorm = 21.5107, GNorm = 3.8133, lr_0 = 9.3925e-04
Validation auc = 0.686854
Epoch 45
Validation auc = 0.687324
Epoch 46
Validation auc = 0.694836
Epoch 47
Validation auc = 0.708920
Epoch 48
Validation auc = 0.721127
Epoch 49
Loss = 2.9143e-01, PNorm = 21.5938, GNorm = 3.4563, lr_0 = 9.3122e-04
Validation auc = 0.722066
Epoch 50
Validation auc = 0.714085
Epoch 51
Validation auc = 0.712676
Epoch 52
Validation auc = 0.713615
Epoch 53
Validation auc = 0.706573
Epoch 54
Loss = 3.9702e-01, PNorm = 21.6791, GNorm = 3.0889, lr_0 = 9.2326e-04
Validation auc = 0.694836
Epoch 55
Validation auc = 0.686385
Epoch 56
Validation auc = 0.677934
Epoch 57
Validation auc = 0.666667
Epoch 58
Validation auc = 0.665258
Epoch 59
Loss = 3.4870e-01, PNorm = 21.7552, GNorm = 1.6231, lr_0 = 9.1537e-04
Validation auc = 0.665728
Epoch 60
Validation auc = 0.657277
Epoch 61
Validation auc = 0.655399
Epoch 62
Validation auc = 0.659624
Epoch 63
Validation auc = 0.665258
Epoch 64
Loss = 3.0284e-01, PNorm = 21.8441, GNorm = 3.4965, lr_0 = 9.0754e-04
Validation auc = 0.670892
Epoch 65
Validation auc = 0.674648
Epoch 66
Validation auc = 0.678873
Epoch 67
Validation auc = 0.679812
Epoch 68
Validation auc = 0.686385
Epoch 69
Loss = 4.1420e-01, PNorm = 21.9325, GNorm = 7.1938, lr_0 = 8.9978e-04
Validation auc = 0.679812
Epoch 70
Validation auc = 0.682160
Epoch 71
Validation auc = 0.677934
Epoch 72
Validation auc = 0.672770
Epoch 73
Validation auc = 0.693897
Epoch 74
Loss = 2.9796e-01, PNorm = 22.0219, GNorm = 4.2246, lr_0 = 8.9209e-04
Validation auc = 0.699061
Epoch 75
Validation auc = 0.706103
Epoch 76
Validation auc = 0.707981
Epoch 77
Validation auc = 0.705164
Epoch 78
Validation auc = 0.704695
Epoch 79
Loss = 2.7874e-01, PNorm = 22.1068, GNorm = 5.0070, lr_0 = 8.8447e-04
Validation auc = 0.702817
Epoch 80
Validation auc = 0.700469
Epoch 81
Validation auc = 0.701408
Epoch 82
Validation auc = 0.695775
Epoch 83
Validation auc = 0.698122
Epoch 84
Loss = 2.4901e-01, PNorm = 22.1927, GNorm = 1.9110, lr_0 = 8.7691e-04
Validation auc = 0.701408
Epoch 85
Validation auc = 0.703286
Epoch 86
Validation auc = 0.699061
Epoch 87
Validation auc = 0.694366
Epoch 88
Validation auc = 0.689671
Epoch 89
Loss = 3.2631e-01, PNorm = 22.2740, GNorm = 4.4958, lr_0 = 8.6941e-04
Validation auc = 0.684038
Epoch 90
Validation auc = 0.684038
Epoch 91
Validation auc = 0.683568
Epoch 92
Validation auc = 0.687793
Epoch 93
Validation auc = 0.681690
Epoch 94
Loss = 2.5277e-01, PNorm = 22.3500, GNorm = 1.7089, lr_0 = 8.6198e-04
Validation auc = 0.678404
Epoch 95
Validation auc = 0.676995
Epoch 96
Validation auc = 0.670892
Epoch 97
Validation auc = 0.670423
Epoch 98
Validation auc = 0.671362
Epoch 99
Loss = 2.5922e-01, PNorm = 22.4353, GNorm = 2.8076, lr_0 = 8.5461e-04
Validation auc = 0.673239
Epoch 100
Validation auc = 0.676995
Epoch 101
Validation auc = 0.676526
Epoch 102
Validation auc = 0.681221
Epoch 103
Validation auc = 0.687324
Epoch 104
Loss = 2.4377e-01, PNorm = 22.5249, GNorm = 4.1541, lr_0 = 8.4730e-04
Validation auc = 0.681221
Epoch 105
Validation auc = 0.676526
Epoch 106
Validation auc = 0.685915
Epoch 107
Validation auc = 0.692488
Epoch 108
Validation auc = 0.696244
Epoch 109
Loss = 2.1962e-01, PNorm = 22.6013, GNorm = 1.8612, lr_0 = 8.4006e-04
Validation auc = 0.700469
Epoch 110
Validation auc = 0.706573
Epoch 111
Validation auc = 0.704225
Epoch 112
Validation auc = 0.704225
Epoch 113
Validation auc = 0.703756
Epoch 114
Loss = 2.2915e-01, PNorm = 22.6865, GNorm = 1.3611, lr_0 = 8.3288e-04
Validation auc = 0.700469
Epoch 115
Validation auc = 0.698592
Epoch 116
Validation auc = 0.696244
Epoch 117
Validation auc = 0.696244
Epoch 118
Validation auc = 0.695775
Epoch 119
Loss = 2.6323e-01, PNorm = 22.7665, GNorm = 3.6265, lr_0 = 8.2576e-04
Validation auc = 0.700000
Epoch 120
Validation auc = 0.705634
Epoch 121
Validation auc = 0.705164
Epoch 122
Validation auc = 0.709859
Epoch 123
Validation auc = 0.706573
Epoch 124
Loss = 2.1438e-01, PNorm = 22.8451, GNorm = 1.7414, lr_0 = 8.1870e-04
Validation auc = 0.704225
Epoch 125
Validation auc = 0.703286
Epoch 126
Validation auc = 0.697653
Epoch 127
Validation auc = 0.693897
Epoch 128
Validation auc = 0.688263
Epoch 129
Loss = 2.1886e-01, PNorm = 22.9256, GNorm = 2.2957, lr_0 = 8.1170e-04
Validation auc = 0.682160
Epoch 130
Validation auc = 0.666667
Epoch 131
Validation auc = 0.645540
Epoch 132
Validation auc = 0.642254
Epoch 133
Validation auc = 0.682629
Epoch 134
Loss = 2.5843e-01, PNorm = 22.9968, GNorm = 5.3159, lr_0 = 8.0476e-04
Validation auc = 0.687324
Epoch 135
Validation auc = 0.686385
Epoch 136
Validation auc = 0.686385
Epoch 137
Validation auc = 0.686385
Epoch 138
Validation auc = 0.681221
Epoch 139
Loss = 2.1105e-01, PNorm = 23.0685, GNorm = 1.2728, lr_0 = 7.9788e-04
Validation auc = 0.678873
Epoch 140
Validation auc = 0.676526
Epoch 141
Validation auc = 0.676526
Epoch 142
Validation auc = 0.677465
Epoch 143
Validation auc = 0.688263
Epoch 144
Loss = 2.4005e-01, PNorm = 23.1418, GNorm = 2.3458, lr_0 = 7.9106e-04
Validation auc = 0.693427
Epoch 145
Validation auc = 0.698592
Epoch 146
Validation auc = 0.702817
Epoch 147
Validation auc = 0.703756
Epoch 148
Validation auc = 0.704225
Epoch 149
Loss = 2.5569e-01, PNorm = 23.2032, GNorm = 1.6983, lr_0 = 7.8430e-04
Validation auc = 0.706573
Epoch 150
Validation auc = 0.706573
Epoch 151
Validation auc = 0.704225
Epoch 152
Validation auc = 0.700000
Epoch 153
Validation auc = 0.700000
Epoch 154
Loss = 1.8596e-01, PNorm = 23.2692, GNorm = 3.4705, lr_0 = 7.7759e-04
Validation auc = 0.696714
Epoch 155
Validation auc = 0.688732
Epoch 156
Validation auc = 0.689202
Epoch 157
Validation auc = 0.689202
Epoch 158
Validation auc = 0.693427
Epoch 159
Loss = 2.0030e-01, PNorm = 23.3348, GNorm = 1.7236, lr_0 = 7.7095e-04
Validation auc = 0.685915
Epoch 160
Validation auc = 0.685446
Epoch 161
Validation auc = 0.689671
Epoch 162
Validation auc = 0.692019
Epoch 163
Validation auc = 0.691549
Epoch 164
Loss = 2.2659e-01, PNorm = 23.3958, GNorm = 4.0205, lr_0 = 7.6436e-04
Validation auc = 0.687793
Epoch 165
Validation auc = 0.681690
Epoch 166
Validation auc = 0.681221
Epoch 167
Validation auc = 0.679812
Epoch 168
Validation auc = 0.671362
Epoch 169
Loss = 2.2204e-01, PNorm = 23.4655, GNorm = 1.3710, lr_0 = 7.5782e-04
Validation auc = 0.654460
Epoch 170
Validation auc = 0.653521
Epoch 171
Validation auc = 0.648826
Epoch 172
Validation auc = 0.646479
Epoch 173
Validation auc = 0.643662
Epoch 174
Loss = 2.1994e-01, PNorm = 23.5318, GNorm = 1.8763, lr_0 = 7.5134e-04
Validation auc = 0.642254
Epoch 175
Validation auc = 0.656808
Epoch 176
Validation auc = 0.676056
Epoch 177
Validation auc = 0.675587
Epoch 178
Validation auc = 0.679343
Epoch 179
Loss = 1.6662e-01, PNorm = 23.5982, GNorm = 1.3766, lr_0 = 7.4492e-04
Validation auc = 0.682629
Epoch 180
Validation auc = 0.686385
Epoch 181
Validation auc = 0.686385
Epoch 182
Validation auc = 0.684977
Epoch 183
Validation auc = 0.687324
Epoch 184
Loss = 1.8840e-01, PNorm = 23.6721, GNorm = 1.2902, lr_0 = 7.3855e-04
Validation auc = 0.688263
Epoch 185
Validation auc = 0.684507
Epoch 186
Validation auc = 0.683568
Epoch 187
Validation auc = 0.684038
Epoch 188
Validation auc = 0.684038
Epoch 189
Loss = 2.6031e-01, PNorm = 23.7383, GNorm = 2.6061, lr_0 = 7.3224e-04
Validation auc = 0.688263
Epoch 190
Validation auc = 0.686854
Epoch 191
Validation auc = 0.678404
Epoch 192
Validation auc = 0.684038
Epoch 193
Validation auc = 0.683568
Epoch 194
Loss = 1.9431e-01, PNorm = 23.8065, GNorm = 1.4847, lr_0 = 7.2598e-04
Validation auc = 0.677465
Epoch 195
Validation auc = 0.682160
Epoch 196
Validation auc = 0.688263
Epoch 197
Validation auc = 0.690610
Epoch 198
Validation auc = 0.687324
Epoch 199
Loss = 1.4518e-01, PNorm = 23.8726, GNorm = 2.0998, lr_0 = 7.1977e-04
Validation auc = 0.686385
Epoch 200
Validation auc = 0.685446
Epoch 201
Validation auc = 0.687324
Epoch 202
Validation auc = 0.689202
Epoch 203
Validation auc = 0.689202
Epoch 204
Loss = 1.6827e-01, PNorm = 23.9357, GNorm = 1.4197, lr_0 = 7.1362e-04
Validation auc = 0.690141
Epoch 205
Validation auc = 0.690610
Epoch 206
Validation auc = 0.691549
Epoch 207
Validation auc = 0.690610
Epoch 208
Validation auc = 0.684038
Epoch 209
Loss = 1.6723e-01, PNorm = 24.0040, GNorm = 2.9619, lr_0 = 7.0752e-04
Validation auc = 0.678873
Epoch 210
Validation auc = 0.676526
Epoch 211
Validation auc = 0.677934
Epoch 212
Validation auc = 0.676056
Epoch 213
Validation auc = 0.672770
Epoch 214
Loss = 1.1171e-01, PNorm = 24.0656, GNorm = 0.8306, lr_0 = 7.0147e-04
Validation auc = 0.670423
Epoch 215
Validation auc = 0.671362
Epoch 216
Validation auc = 0.678404
Epoch 217
Validation auc = 0.684977
Epoch 218
Validation auc = 0.685446
Epoch 219
Loss = 1.5862e-01, PNorm = 24.1263, GNorm = 2.5060, lr_0 = 6.9548e-04
Validation auc = 0.684977
Epoch 220
Validation auc = 0.682160
Epoch 221
Validation auc = 0.685446
Epoch 222
Validation auc = 0.685915
Epoch 223
Validation auc = 0.686854
Epoch 224
Loss = 1.7072e-01, PNorm = 24.1878, GNorm = 1.7551, lr_0 = 6.8953e-04
Validation auc = 0.681221
Epoch 225
Validation auc = 0.679343
Epoch 226
Validation auc = 0.674648
Epoch 227
Validation auc = 0.678873
Epoch 228
Validation auc = 0.672770
Epoch 229
Loss = 1.7622e-01, PNorm = 24.2481, GNorm = 1.5529, lr_0 = 6.8364e-04
Validation auc = 0.669953
Epoch 230
Validation auc = 0.665728
Epoch 231
Validation auc = 0.661972
Epoch 232
Validation auc = 0.665258
Epoch 233
Validation auc = 0.663850
Epoch 234
Loss = 1.2473e-01, PNorm = 24.2999, GNorm = 1.0300, lr_0 = 6.7779e-04
Validation auc = 0.667606
Epoch 235
Validation auc = 0.672770
Epoch 236
Validation auc = 0.676056
Epoch 237
Validation auc = 0.676995
Epoch 238
Validation auc = 0.672300
Epoch 239
Loss = 1.5973e-01, PNorm = 24.3518, GNorm = 2.6832, lr_0 = 6.7200e-04
Validation auc = 0.667136
Epoch 240
Validation auc = 0.670423
Epoch 241
Validation auc = 0.670423
Epoch 242
Validation auc = 0.669014
Epoch 243
Validation auc = 0.673709
Epoch 244
Loss = 1.8732e-01, PNorm = 24.4185, GNorm = 3.7242, lr_0 = 6.6625e-04
Validation auc = 0.679812
Epoch 245
Validation auc = 0.686854
Epoch 246
Validation auc = 0.683099
Epoch 247
Validation auc = 0.678404
Epoch 248
Validation auc = 0.675117
Epoch 249
Loss = 1.3490e-01, PNorm = 24.4843, GNorm = 1.0037, lr_0 = 6.6056e-04
Validation auc = 0.672770
Epoch 250
Validation auc = 0.664319
Epoch 251
Validation auc = 0.651643
Epoch 252
Validation auc = 0.645070
Epoch 253
Validation auc = 0.645070
Epoch 254
Loss = 1.8257e-01, PNorm = 24.5487, GNorm = 3.1421, lr_0 = 6.5491e-04
Validation auc = 0.665258
Epoch 255
Validation auc = 0.682629
Epoch 256
Validation auc = 0.686385
Epoch 257
Validation auc = 0.678873
Epoch 258
Validation auc = 0.676526
Epoch 259
Loss = 1.9999e-01, PNorm = 24.6105, GNorm = 2.2792, lr_0 = 6.4931e-04
Validation auc = 0.687793
Epoch 260
Validation auc = 0.691549
Epoch 261
Validation auc = 0.688732
Epoch 262
Validation auc = 0.686854
Epoch 263
Validation auc = 0.686854
Epoch 264
Loss = 1.1485e-01, PNorm = 24.6711, GNorm = 2.5232, lr_0 = 6.4376e-04
Validation auc = 0.689202
Epoch 265
Validation auc = 0.686854
Epoch 266
Validation auc = 0.690610
Epoch 267
Validation auc = 0.688263
Epoch 268
Validation auc = 0.684507
Epoch 269
Loss = 1.4280e-01, PNorm = 24.7352, GNorm = 1.0839, lr_0 = 6.3826e-04
Validation auc = 0.677465
Epoch 270
Validation auc = 0.676995
Epoch 271
Validation auc = 0.680751
Epoch 272
Validation auc = 0.679343
Epoch 273
Validation auc = 0.673709
Epoch 274
Loss = 1.5029e-01, PNorm = 24.7901, GNorm = 2.5717, lr_0 = 6.3280e-04
Validation auc = 0.667606
Epoch 275
Validation auc = 0.658216
Epoch 276
Validation auc = 0.650704
Epoch 277
Validation auc = 0.644601
Epoch 278
Validation auc = 0.641784
Epoch 279
Loss = 1.6248e-01, PNorm = 24.8480, GNorm = 1.0055, lr_0 = 6.2739e-04
Validation auc = 0.627700
Epoch 280
Validation auc = 0.624413
Epoch 281
Validation auc = 0.621127
Epoch 282
Validation auc = 0.622066
Epoch 283
Validation auc = 0.638028
Epoch 284
Loss = 1.4332e-01, PNorm = 24.9046, GNorm = 2.3372, lr_0 = 6.2203e-04
Validation auc = 0.644131
Epoch 285
Validation auc = 0.645070
Epoch 286
Validation auc = 0.648357
Epoch 287
Validation auc = 0.656808
Epoch 288
Validation auc = 0.659624
Epoch 289
Loss = 1.4929e-01, PNorm = 24.9597, GNorm = 1.5883, lr_0 = 6.1671e-04
Validation auc = 0.670423
Epoch 290
Validation auc = 0.674648
Epoch 291
Validation auc = 0.677934
Epoch 292
Validation auc = 0.680282
Epoch 293
Validation auc = 0.680751
Epoch 294
Loss = 2.2587e-01, PNorm = 25.0108, GNorm = 1.9508, lr_0 = 6.1144e-04
Validation auc = 0.687324
Epoch 295
Validation auc = 0.688732
Epoch 296
Validation auc = 0.685446
Epoch 297
Validation auc = 0.684507
Epoch 298
Validation auc = 0.684977
Epoch 299
Loss = 1.6240e-01, PNorm = 25.0667, GNorm = 1.3419, lr_0 = 6.0621e-04
Validation auc = 0.686385
Model 0 best validation auc = 0.731455 on epoch 16
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.874774
Ensemble test auc = 0.874774
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.519249
Epoch 1
Validation auc = 0.593427
Epoch 2
Validation auc = 0.632394
Epoch 3
Validation auc = 0.656808
Epoch 4
Loss = 7.0717e-01, PNorm = 20.9834, GNorm = 5.2813, lr_0 = 6.5000e-04
Validation auc = 0.672770
Epoch 5
Validation auc = 0.669014
Epoch 6
Validation auc = 0.659624
Epoch 7
Validation auc = 0.652113
Epoch 8
Validation auc = 0.635681
Epoch 9
Loss = 6.3960e-01, PNorm = 21.0300, GNorm = 3.5011, lr_0 = 9.9743e-04
Validation auc = 0.621596
Epoch 10
Validation auc = 0.629577
Epoch 11
Validation auc = 0.628638
Epoch 12
Validation auc = 0.673239
Epoch 13
Validation auc = 0.668075
Epoch 14
Loss = 4.6437e-01, PNorm = 21.0948, GNorm = 2.7550, lr_0 = 9.8890e-04
Validation auc = 0.675117
Epoch 15
Validation auc = 0.689202
Epoch 16
Validation auc = 0.680282
Epoch 17
Validation auc = 0.658685
Epoch 18
Validation auc = 0.661972
Epoch 19
Loss = 4.1016e-01, PNorm = 21.1611, GNorm = 4.4894, lr_0 = 9.8045e-04
Validation auc = 0.643192
Epoch 20
Validation auc = 0.646009
Epoch 21
Validation auc = 0.678404
Epoch 22
Validation auc = 0.702347
Epoch 23
Validation auc = 0.703286
Epoch 24
Loss = 4.7781e-01, PNorm = 21.2337, GNorm = 8.0701, lr_0 = 9.7207e-04
Validation auc = 0.707981
Epoch 25
Validation auc = 0.705634
Epoch 26
Validation auc = 0.701408
Epoch 27
Validation auc = 0.699061
Epoch 28
Validation auc = 0.694366
Epoch 29
Loss = 3.7654e-01, PNorm = 21.3019, GNorm = 3.4270, lr_0 = 9.6376e-04
Validation auc = 0.706573
Epoch 30
Validation auc = 0.706573
Epoch 31
Validation auc = 0.705164
Epoch 32
Validation auc = 0.710798
Epoch 33
Validation auc = 0.712207
Epoch 34
Loss = 3.2497e-01, PNorm = 21.3709, GNorm = 3.3165, lr_0 = 9.5552e-04
Validation auc = 0.711268
Epoch 35
Validation auc = 0.708451
Epoch 36
Validation auc = 0.702817
Epoch 37
Validation auc = 0.697653
Epoch 38
Validation auc = 0.698592
Epoch 39
Loss = 4.0199e-01, PNorm = 21.4446, GNorm = 5.5697, lr_0 = 9.4735e-04
Validation auc = 0.694836
Epoch 40
Validation auc = 0.686385
Epoch 41
Validation auc = 0.670892
Epoch 42
Validation auc = 0.665258
Epoch 43
Validation auc = 0.675587
Epoch 44
Loss = 4.3419e-01, PNorm = 21.5149, GNorm = 1.6827, lr_0 = 9.3925e-04
Validation auc = 0.684977
Epoch 45
Validation auc = 0.683099
Epoch 46
Validation auc = 0.683099
Epoch 47
Validation auc = 0.684038
Epoch 48
Validation auc = 0.682629
Epoch 49
Loss = 3.5491e-01, PNorm = 21.5837, GNorm = 2.1034, lr_0 = 9.3122e-04
Validation auc = 0.662441
Epoch 50
Validation auc = 0.653991
Epoch 51
Validation auc = 0.661502
Epoch 52
Validation auc = 0.649296
Epoch 53
Validation auc = 0.633803
Epoch 54
Loss = 2.9333e-01, PNorm = 21.6531, GNorm = 2.5937, lr_0 = 9.2326e-04
Validation auc = 0.630047
Epoch 55
Validation auc = 0.655869
Epoch 56
Validation auc = 0.669484
Epoch 57
Validation auc = 0.658216
Epoch 58
Validation auc = 0.668075
Epoch 59
Loss = 3.4492e-01, PNorm = 21.7220, GNorm = 4.0137, lr_0 = 9.1537e-04
Validation auc = 0.674648
Epoch 60
Validation auc = 0.669953
Epoch 61
Validation auc = 0.680282
Epoch 62
Validation auc = 0.697653
Epoch 63
Validation auc = 0.701408
Epoch 64
Loss = 2.6372e-01, PNorm = 21.7966, GNorm = 2.4313, lr_0 = 9.0754e-04
Validation auc = 0.704225
Epoch 65
Validation auc = 0.705164
Epoch 66
Validation auc = 0.707981
Epoch 67
Validation auc = 0.705164
Epoch 68
Validation auc = 0.707042
Epoch 69
Loss = 2.5724e-01, PNorm = 21.8748, GNorm = 1.5039, lr_0 = 8.9978e-04
Validation auc = 0.709859
Epoch 70
Validation auc = 0.714554
Epoch 71
Validation auc = 0.711737
Epoch 72
Validation auc = 0.708451
Epoch 73
Validation auc = 0.705634
Epoch 74
Loss = 2.5882e-01, PNorm = 21.9494, GNorm = 2.8789, lr_0 = 8.9209e-04
Validation auc = 0.706573
Epoch 75
Validation auc = 0.703756
Epoch 76
Validation auc = 0.699531
Epoch 77
Validation auc = 0.697183
Epoch 78
Validation auc = 0.696244
Epoch 79
Loss = 3.7710e-01, PNorm = 22.0300, GNorm = 2.2812, lr_0 = 8.8447e-04
Validation auc = 0.706103
Epoch 80
Validation auc = 0.715493
Epoch 81
Validation auc = 0.720657
Epoch 82
Validation auc = 0.716901
Epoch 83
Validation auc = 0.715023
Epoch 84
Loss = 3.9992e-01, PNorm = 22.0992, GNorm = 3.6044, lr_0 = 8.7691e-04
Validation auc = 0.714554
Epoch 85
Validation auc = 0.712676
Epoch 86
Validation auc = 0.707981
Epoch 87
Validation auc = 0.703286
Epoch 88
Validation auc = 0.696714
Epoch 89
Loss = 2.8645e-01, PNorm = 22.1804, GNorm = 2.6883, lr_0 = 8.6941e-04
Validation auc = 0.697653
Epoch 90
Validation auc = 0.693897
Epoch 91
Validation auc = 0.683099
Epoch 92
Validation auc = 0.671831
Epoch 93
Validation auc = 0.683568
Epoch 94
Loss = 2.4020e-01, PNorm = 22.2612, GNorm = 3.3803, lr_0 = 8.6198e-04
Validation auc = 0.684977
Epoch 95
Validation auc = 0.679343
Epoch 96
Validation auc = 0.682629
Epoch 97
Validation auc = 0.684507
Epoch 98
Validation auc = 0.684038
Epoch 99
Loss = 2.2959e-01, PNorm = 22.3417, GNorm = 1.8314, lr_0 = 8.5461e-04
Validation auc = 0.690610
Epoch 100
Validation auc = 0.693427
Epoch 101
Validation auc = 0.694366
Epoch 102
Validation auc = 0.692958
Epoch 103
Validation auc = 0.684038
Epoch 104
Loss = 2.4457e-01, PNorm = 22.4088, GNorm = 1.5551, lr_0 = 8.4730e-04
Validation auc = 0.676056
Epoch 105
Validation auc = 0.669953
Epoch 106
Validation auc = 0.669484
Epoch 107
Validation auc = 0.670423
Epoch 108
Validation auc = 0.666667
Epoch 109
Loss = 3.0553e-01, PNorm = 22.4951, GNorm = 3.8977, lr_0 = 8.4006e-04
Validation auc = 0.661972
Epoch 110
Validation auc = 0.669484
Epoch 111
Validation auc = 0.669014
Epoch 112
Validation auc = 0.668545
Epoch 113
Validation auc = 0.671362
Epoch 114
Loss = 2.3892e-01, PNorm = 22.5740, GNorm = 1.6609, lr_0 = 8.3288e-04
Validation auc = 0.668545
Epoch 115
Validation auc = 0.674178
Epoch 116
Validation auc = 0.678873
Epoch 117
Validation auc = 0.671831
Epoch 118
Validation auc = 0.674648
Epoch 119
Loss = 3.2376e-01, PNorm = 22.6517, GNorm = 5.5609, lr_0 = 8.2576e-04
Validation auc = 0.684977
Epoch 120
Validation auc = 0.690610
Epoch 121
Validation auc = 0.691549
Epoch 122
Validation auc = 0.688263
Epoch 123
Validation auc = 0.687793
Epoch 124
Loss = 2.3147e-01, PNorm = 22.7514, GNorm = 4.5428, lr_0 = 8.1870e-04
Validation auc = 0.681221
Epoch 125
Validation auc = 0.680751
Epoch 126
Validation auc = 0.675117
Epoch 127
Validation auc = 0.669014
Epoch 128
Validation auc = 0.669014
Epoch 129
Loss = 2.0574e-01, PNorm = 22.8464, GNorm = 3.6786, lr_0 = 8.1170e-04
Validation auc = 0.666667
Epoch 130
Validation auc = 0.665258
Epoch 131
Validation auc = 0.671831
Epoch 132
Validation auc = 0.673239
Epoch 133
Validation auc = 0.663380
Epoch 134
Loss = 1.6659e-01, PNorm = 22.9285, GNorm = 2.6473, lr_0 = 8.0476e-04
Validation auc = 0.650235
Epoch 135
Validation auc = 0.661033
Epoch 136
Validation auc = 0.653521
Epoch 137
Validation auc = 0.626761
Epoch 138
Validation auc = 0.626291
Epoch 139
Loss = 2.8908e-01, PNorm = 23.0093, GNorm = 1.5586, lr_0 = 7.9788e-04
Validation auc = 0.655869
Epoch 140
Validation auc = 0.661502
Epoch 141
Validation auc = 0.665728
Epoch 142
Validation auc = 0.662911
Epoch 143
Validation auc = 0.670892
Epoch 144
Loss = 2.8270e-01, PNorm = 23.0909, GNorm = 1.8565, lr_0 = 7.9106e-04
Validation auc = 0.669484
Epoch 145
Validation auc = 0.674178
Epoch 146
Validation auc = 0.667136
Epoch 147
Validation auc = 0.677465
Epoch 148
Validation auc = 0.676995
Epoch 149
Loss = 1.9025e-01, PNorm = 23.1664, GNorm = 2.2305, lr_0 = 7.8430e-04
Validation auc = 0.680751
Epoch 150
Validation auc = 0.674178
Epoch 151
Validation auc = 0.673239
Epoch 152
Validation auc = 0.678873
Epoch 153
Validation auc = 0.669484
Epoch 154
Loss = 2.2263e-01, PNorm = 23.2409, GNorm = 1.4477, lr_0 = 7.7759e-04
Validation auc = 0.665258
Epoch 155
Validation auc = 0.660094
Epoch 156
Validation auc = 0.665728
Epoch 157
Validation auc = 0.658685
Epoch 158
Validation auc = 0.653991
Epoch 159
Loss = 2.5061e-01, PNorm = 23.3151, GNorm = 1.5320, lr_0 = 7.7095e-04
Validation auc = 0.669953
Epoch 160
Validation auc = 0.677934
Epoch 161
Validation auc = 0.676995
Epoch 162
Validation auc = 0.677465
Epoch 163
Validation auc = 0.679343
Epoch 164
Loss = 2.0997e-01, PNorm = 23.3825, GNorm = 1.7686, lr_0 = 7.6436e-04
Validation auc = 0.678404
Epoch 165
Validation auc = 0.676526
Epoch 166
Validation auc = 0.671831
Epoch 167
Validation auc = 0.671362
Epoch 168
Validation auc = 0.668075
Epoch 169
Loss = 1.7137e-01, PNorm = 23.4523, GNorm = 2.4292, lr_0 = 7.5782e-04
Validation auc = 0.671831
Epoch 170
Validation auc = 0.663850
Epoch 171
Validation auc = 0.663850
Epoch 172
Validation auc = 0.666197
Epoch 173
Validation auc = 0.678404
Epoch 174
Loss = 2.3333e-01, PNorm = 23.5203, GNorm = 2.3949, lr_0 = 7.5134e-04
Validation auc = 0.676056
Epoch 175
Validation auc = 0.671362
Epoch 176
Validation auc = 0.666197
Epoch 177
Validation auc = 0.662911
Epoch 178
Validation auc = 0.662911
Epoch 179
Loss = 2.2225e-01, PNorm = 23.5965, GNorm = 2.7708, lr_0 = 7.4492e-04
Validation auc = 0.667136
Epoch 180
Validation auc = 0.670423
Epoch 181
Validation auc = 0.666667
Epoch 182
Validation auc = 0.669014
Epoch 183
Validation auc = 0.668545
Epoch 184
Loss = 2.9595e-01, PNorm = 23.6740, GNorm = 3.8856, lr_0 = 7.3855e-04
Validation auc = 0.670892
Epoch 185
Validation auc = 0.672300
Epoch 186
Validation auc = 0.658685
Epoch 187
Validation auc = 0.664789
Epoch 188
Validation auc = 0.668075
Epoch 189
Loss = 1.8911e-01, PNorm = 23.7420, GNorm = 2.0559, lr_0 = 7.3224e-04
Validation auc = 0.666667
Epoch 190
Validation auc = 0.669953
Epoch 191
Validation auc = 0.689202
Epoch 192
Validation auc = 0.698122
Epoch 193
Validation auc = 0.697653
Epoch 194
Loss = 1.9708e-01, PNorm = 23.8086, GNorm = 1.6140, lr_0 = 7.2598e-04
Validation auc = 0.693897
Epoch 195
Validation auc = 0.693427
Epoch 196
Validation auc = 0.689202
Epoch 197
Validation auc = 0.686854
Epoch 198
Validation auc = 0.677465
Epoch 199
Loss = 2.3644e-01, PNorm = 23.8721, GNorm = 3.7521, lr_0 = 7.1977e-04
Validation auc = 0.679343
Epoch 200
Validation auc = 0.676995
Epoch 201
Validation auc = 0.672300
Epoch 202
Validation auc = 0.671362
Epoch 203
Validation auc = 0.670423
Epoch 204
Loss = 2.0933e-01, PNorm = 23.9398, GNorm = 3.0344, lr_0 = 7.1362e-04
Validation auc = 0.672300
Epoch 205
Validation auc = 0.669484
Epoch 206
Validation auc = 0.665258
Epoch 207
Validation auc = 0.663850
Epoch 208
Validation auc = 0.661033
Epoch 209
Loss = 1.1528e-01, PNorm = 24.0008, GNorm = 0.9859, lr_0 = 7.0752e-04
Validation auc = 0.658685
Epoch 210
Validation auc = 0.657746
Epoch 211
Validation auc = 0.659155
Epoch 212
Validation auc = 0.657277
Epoch 213
Validation auc = 0.651643
Epoch 214
Loss = 2.0059e-01, PNorm = 24.0585, GNorm = 2.8599, lr_0 = 7.0147e-04
Validation auc = 0.639437
Epoch 215
Validation auc = 0.638967
Epoch 216
Validation auc = 0.646009
Epoch 217
Validation auc = 0.658216
Epoch 218
Validation auc = 0.664319
Epoch 219
Loss = 1.1866e-01, PNorm = 24.1081, GNorm = 1.9495, lr_0 = 6.9548e-04
Validation auc = 0.666197
Epoch 220
Validation auc = 0.676056
Epoch 221
Validation auc = 0.676995
Epoch 222
Validation auc = 0.672770
Epoch 223
Validation auc = 0.668545
Epoch 224
Loss = 1.8301e-01, PNorm = 24.1656, GNorm = 3.5261, lr_0 = 6.8953e-04
Validation auc = 0.669484
Epoch 225
Validation auc = 0.670892
Epoch 226
Validation auc = 0.671831
Epoch 227
Validation auc = 0.670892
Epoch 228
Validation auc = 0.674648
Epoch 229
Loss = 1.2757e-01, PNorm = 24.2293, GNorm = 1.5634, lr_0 = 6.8364e-04
Validation auc = 0.676526
Epoch 230
Validation auc = 0.677934
Epoch 231
Validation auc = 0.687324
Epoch 232
Validation auc = 0.688263
Epoch 233
Validation auc = 0.688263
Epoch 234
Loss = 1.7938e-01, PNorm = 24.2961, GNorm = 5.0257, lr_0 = 6.7779e-04
Validation auc = 0.684038
Epoch 235
Validation auc = 0.692488
Epoch 236
Validation auc = 0.689202
Epoch 237
Validation auc = 0.693427
Epoch 238
Validation auc = 0.693897
Epoch 239
Loss = 2.0436e-01, PNorm = 24.3639, GNorm = 2.9153, lr_0 = 6.7200e-04
Validation auc = 0.691549
Epoch 240
Validation auc = 0.691549
Epoch 241
Validation auc = 0.688263
Epoch 242
Validation auc = 0.682629
Epoch 243
Validation auc = 0.682160
Epoch 244
Loss = 1.7367e-01, PNorm = 24.4237, GNorm = 1.7099, lr_0 = 6.6625e-04
Validation auc = 0.683099
Epoch 245
Validation auc = 0.682160
Epoch 246
Validation auc = 0.682160
Epoch 247
Validation auc = 0.683568
Epoch 248
Validation auc = 0.686854
Epoch 249
Loss = 1.5866e-01, PNorm = 24.4779, GNorm = 3.0769, lr_0 = 6.6056e-04
Validation auc = 0.689202
Epoch 250
Validation auc = 0.689671
Epoch 251
Validation auc = 0.687793
Epoch 252
Validation auc = 0.685915
Epoch 253
Validation auc = 0.684507
Epoch 254
Loss = 1.5106e-01, PNorm = 24.5393, GNorm = 2.7788, lr_0 = 6.5491e-04
Validation auc = 0.677934
Epoch 255
Validation auc = 0.668545
Epoch 256
Validation auc = 0.663380
Epoch 257
Validation auc = 0.661972
Epoch 258
Validation auc = 0.661972
Epoch 259
Loss = 1.2637e-01, PNorm = 24.6092, GNorm = 1.0472, lr_0 = 6.4931e-04
Validation auc = 0.665258
Epoch 260
Validation auc = 0.667136
Epoch 261
Validation auc = 0.674178
Epoch 262
Validation auc = 0.674178
Epoch 263
Validation auc = 0.672770
Epoch 264
Loss = 9.9380e-02, PNorm = 24.6633, GNorm = 1.6386, lr_0 = 6.4376e-04
Validation auc = 0.670892
Epoch 265
Validation auc = 0.671362
Epoch 266
Validation auc = 0.666197
Epoch 267
Validation auc = 0.668075
Epoch 268
Validation auc = 0.673709
Epoch 269
Loss = 2.0966e-01, PNorm = 24.7189, GNorm = 1.5576, lr_0 = 6.3826e-04
Validation auc = 0.673709
Epoch 270
Validation auc = 0.669484
Epoch 271
Validation auc = 0.667606
Epoch 272
Validation auc = 0.670423
Epoch 273
Validation auc = 0.671362
Epoch 274
Loss = 1.1010e-01, PNorm = 24.7782, GNorm = 3.2108, lr_0 = 6.3280e-04
Validation auc = 0.665728
Epoch 275
Validation auc = 0.662441
Epoch 276
Validation auc = 0.664789
Epoch 277
Validation auc = 0.662911
Epoch 278
Validation auc = 0.663380
Epoch 279
Loss = 1.7857e-01, PNorm = 24.8312, GNorm = 1.7773, lr_0 = 6.2739e-04
Validation auc = 0.661972
Epoch 280
Validation auc = 0.658685
Epoch 281
Validation auc = 0.654930
Epoch 282
Validation auc = 0.648826
Epoch 283
Validation auc = 0.652113
Epoch 284
Loss = 1.8851e-01, PNorm = 24.8735, GNorm = 4.1507, lr_0 = 6.2203e-04
Validation auc = 0.655399
Epoch 285
Validation auc = 0.660094
Epoch 286
Validation auc = 0.666197
Epoch 287
Validation auc = 0.666667
Epoch 288
Validation auc = 0.663850
Epoch 289
Loss = 1.3574e-01, PNorm = 24.9384, GNorm = 2.4887, lr_0 = 6.1671e-04
Validation auc = 0.659155
Epoch 290
Validation auc = 0.664319
Epoch 291
Validation auc = 0.664789
Epoch 292
Validation auc = 0.666667
Epoch 293
Validation auc = 0.665728
Epoch 294
Loss = 1.6272e-01, PNorm = 25.0027, GNorm = 1.4201, lr_0 = 6.1144e-04
Validation auc = 0.661972
Epoch 295
Validation auc = 0.661972
Epoch 296
Validation auc = 0.662441
Epoch 297
Validation auc = 0.661502
Epoch 298
Validation auc = 0.666197
Epoch 299
Loss = 1.4941e-01, PNorm = 25.0491, GNorm = 1.2584, lr_0 = 6.0621e-04
Validation auc = 0.666667
Model 0 best validation auc = 0.720657 on epoch 81
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.874774
Ensemble test auc = 0.874774
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.487324
Epoch 1
Validation auc = 0.615493
Epoch 2
Validation auc = 0.553052
Epoch 3
Validation auc = 0.563380
Epoch 4
Loss = 7.3356e-01, PNorm = 20.9860, GNorm = 2.5159, lr_0 = 6.5000e-04
Validation auc = 0.587324
Epoch 5
Validation auc = 0.564319
Epoch 6
Validation auc = 0.578404
Epoch 7
Validation auc = 0.649765
Epoch 8
Validation auc = 0.652113
Epoch 9
Loss = 5.6786e-01, PNorm = 21.0324, GNorm = 2.6442, lr_0 = 9.9743e-04
Validation auc = 0.651174
Epoch 10
Validation auc = 0.670423
Epoch 11
Validation auc = 0.681221
Epoch 12
Validation auc = 0.687793
Epoch 13
Validation auc = 0.695305
Epoch 14
Loss = 4.7524e-01, PNorm = 21.1022, GNorm = 3.6549, lr_0 = 9.8890e-04
Validation auc = 0.697183
Epoch 15
Validation auc = 0.706103
Epoch 16
Validation auc = 0.715023
Epoch 17
Validation auc = 0.711268
Epoch 18
Validation auc = 0.720188
Epoch 19
Loss = 3.8114e-01, PNorm = 21.1733, GNorm = 4.7799, lr_0 = 9.8045e-04
Validation auc = 0.727230
Epoch 20
Validation auc = 0.721127
Epoch 21
Validation auc = 0.719718
Epoch 22
Validation auc = 0.720657
Epoch 23
Validation auc = 0.723474
Epoch 24
Loss = 4.1049e-01, PNorm = 21.2446, GNorm = 2.9190, lr_0 = 9.7207e-04
Validation auc = 0.714085
Epoch 25
Validation auc = 0.716432
Epoch 26
Validation auc = 0.710329
Epoch 27
Validation auc = 0.714554
Epoch 28
Validation auc = 0.724883
Epoch 29
Loss = 3.9556e-01, PNorm = 21.3169, GNorm = 6.1104, lr_0 = 9.6376e-04
Validation auc = 0.725352
Epoch 30
Validation auc = 0.723474
Epoch 31
Validation auc = 0.725352
Epoch 32
Validation auc = 0.730047
Epoch 33
Validation auc = 0.726291
Epoch 34
Loss = 2.7721e-01, PNorm = 21.4014, GNorm = 4.1627, lr_0 = 9.5552e-04
Validation auc = 0.723005
Epoch 35
Validation auc = 0.711737
Epoch 36
Validation auc = 0.707042
Epoch 37
Validation auc = 0.702347
Epoch 38
Validation auc = 0.697653
Epoch 39
Loss = 3.3554e-01, PNorm = 21.4882, GNorm = 3.1487, lr_0 = 9.4735e-04
Validation auc = 0.699531
Epoch 40
Validation auc = 0.696244
Epoch 41
Validation auc = 0.700469
Epoch 42
Validation auc = 0.704225
Epoch 43
Validation auc = 0.703756
Epoch 44
Loss = 3.7302e-01, PNorm = 21.5756, GNorm = 1.7727, lr_0 = 9.3925e-04
Validation auc = 0.709390
Epoch 45
Validation auc = 0.707042
Epoch 46
Validation auc = 0.697183
Epoch 47
Validation auc = 0.696714
Epoch 48
Validation auc = 0.702817
Epoch 49
Loss = 3.2446e-01, PNorm = 21.6680, GNorm = 1.6858, lr_0 = 9.3122e-04
Validation auc = 0.696244
Epoch 50
Validation auc = 0.692958
Epoch 51
Validation auc = 0.695775
Epoch 52
Validation auc = 0.689202
Epoch 53
Validation auc = 0.671362
Epoch 54
Loss = 2.6780e-01, PNorm = 21.7494, GNorm = 2.3542, lr_0 = 9.2326e-04
Validation auc = 0.653521
Epoch 55
Validation auc = 0.658216
Epoch 56
Validation auc = 0.660094
Epoch 57
Validation auc = 0.663850
Epoch 58
Validation auc = 0.679343
Epoch 59
Loss = 3.3186e-01, PNorm = 21.8300, GNorm = 4.5426, lr_0 = 9.1537e-04
Validation auc = 0.678873
Epoch 60
Validation auc = 0.678873
Epoch 61
Validation auc = 0.679812
Epoch 62
Validation auc = 0.676526
Epoch 63
Validation auc = 0.675587
Epoch 64
Loss = 2.7873e-01, PNorm = 21.9201, GNorm = 2.4961, lr_0 = 9.0754e-04
Validation auc = 0.664319
Epoch 65
Validation auc = 0.682629
Epoch 66
Validation auc = 0.684507
Epoch 67
Validation auc = 0.678873
Epoch 68
Validation auc = 0.676995
Epoch 69
Loss = 2.6093e-01, PNorm = 21.9959, GNorm = 1.5956, lr_0 = 8.9978e-04
Validation auc = 0.672300
Epoch 70
Validation auc = 0.666197
Epoch 71
Validation auc = 0.655869
Epoch 72
Validation auc = 0.639906
Epoch 73
Validation auc = 0.643662
Epoch 74
Loss = 2.4541e-01, PNorm = 22.0962, GNorm = 1.7709, lr_0 = 8.9209e-04
Validation auc = 0.639906
Epoch 75
Validation auc = 0.658685
Epoch 76
Validation auc = 0.668075
Epoch 77
Validation auc = 0.672770
Epoch 78
Validation auc = 0.671362
Epoch 79
Loss = 1.9988e-01, PNorm = 22.1896, GNorm = 1.4476, lr_0 = 8.8447e-04
Validation auc = 0.659155
Epoch 80
Validation auc = 0.676056
Epoch 81
Validation auc = 0.685446
Epoch 82
Validation auc = 0.690141
Epoch 83
Validation auc = 0.690610
Epoch 84
Loss = 3.2439e-01, PNorm = 22.2743, GNorm = 3.2749, lr_0 = 8.7691e-04
Validation auc = 0.691080
Epoch 85
Validation auc = 0.684977
Epoch 86
Validation auc = 0.669014
Epoch 87
Validation auc = 0.660563
Epoch 88
Validation auc = 0.677934
Epoch 89
Loss = 2.9107e-01, PNorm = 22.3658, GNorm = 2.9620, lr_0 = 8.6941e-04
Validation auc = 0.698122
Epoch 90
Validation auc = 0.706103
Epoch 91
Validation auc = 0.699531
Epoch 92
Validation auc = 0.695775
Epoch 93
Validation auc = 0.691549
Epoch 94
Loss = 2.5654e-01, PNorm = 22.4453, GNorm = 3.6306, lr_0 = 8.6198e-04
Validation auc = 0.685915
Epoch 95
Validation auc = 0.669014
Epoch 96
Validation auc = 0.669014
Epoch 97
Validation auc = 0.672300
Epoch 98
Validation auc = 0.640845
Epoch 99
Loss = 2.9959e-01, PNorm = 22.5380, GNorm = 3.3162, lr_0 = 8.5461e-04
Validation auc = 0.632864
Epoch 100
Validation auc = 0.640376
Epoch 101
Validation auc = 0.648357
Epoch 102
Validation auc = 0.656808
Epoch 103
Validation auc = 0.676995
Epoch 104
Loss = 2.5040e-01, PNorm = 22.6226, GNorm = 2.9388, lr_0 = 8.4730e-04
Validation auc = 0.691080
Epoch 105
Validation auc = 0.687793
Epoch 106
Validation auc = 0.672300
Epoch 107
Validation auc = 0.670423
Epoch 108
Validation auc = 0.653521
Epoch 109
Loss = 2.0992e-01, PNorm = 22.6979, GNorm = 1.9494, lr_0 = 8.4006e-04
Validation auc = 0.642254
Epoch 110
Validation auc = 0.636620
Epoch 111
Validation auc = 0.657277
Epoch 112
Validation auc = 0.675587
Epoch 113
Validation auc = 0.671362
Epoch 114
Loss = 3.0617e-01, PNorm = 22.7737, GNorm = 5.7363, lr_0 = 8.3288e-04
Validation auc = 0.673709
Epoch 115
Validation auc = 0.684038
Epoch 116
Validation auc = 0.681221
Epoch 117
Validation auc = 0.669014
Epoch 118
Validation auc = 0.626291
Epoch 119
Loss = 2.8226e-01, PNorm = 22.8553, GNorm = 2.9404, lr_0 = 8.2576e-04
Validation auc = 0.627230
Epoch 120
Validation auc = 0.644601
Epoch 121
Validation auc = 0.663380
Epoch 122
Validation auc = 0.659155
Epoch 123
Validation auc = 0.676526
Epoch 124
Loss = 3.1762e-01, PNorm = 22.9392, GNorm = 3.4146, lr_0 = 8.1870e-04
Validation auc = 0.680282
Epoch 125
Validation auc = 0.685446
Epoch 126
Validation auc = 0.683099
Epoch 127
Validation auc = 0.679343
Epoch 128
Validation auc = 0.674178
Epoch 129
Loss = 2.7210e-01, PNorm = 23.0236, GNorm = 1.2847, lr_0 = 8.1170e-04
Validation auc = 0.684977
Epoch 130
Validation auc = 0.684977
Epoch 131
Validation auc = 0.684038
Epoch 132
Validation auc = 0.684977
Epoch 133
Validation auc = 0.687793
Epoch 134
Loss = 2.5541e-01, PNorm = 23.1095, GNorm = 3.6168, lr_0 = 8.0476e-04
Validation auc = 0.689671
Epoch 135
Validation auc = 0.694366
Epoch 136
Validation auc = 0.691549
Epoch 137
Validation auc = 0.691080
Epoch 138
Validation auc = 0.687793
Epoch 139
Loss = 2.1705e-01, PNorm = 23.1858, GNorm = 1.3279, lr_0 = 7.9788e-04
Validation auc = 0.681690
Epoch 140
Validation auc = 0.668545
Epoch 141
Validation auc = 0.652113
Epoch 142
Validation auc = 0.631925
Epoch 143
Validation auc = 0.635211
Epoch 144
Loss = 2.2808e-01, PNorm = 23.2628, GNorm = 1.1210, lr_0 = 7.9106e-04
Validation auc = 0.674178
Epoch 145
Validation auc = 0.685915
Epoch 146
Validation auc = 0.690610
Epoch 147
Validation auc = 0.691080
Epoch 148
Validation auc = 0.687324
Epoch 149
Loss = 2.4306e-01, PNorm = 23.3364, GNorm = 4.0934, lr_0 = 7.8430e-04
Validation auc = 0.683099
Epoch 150
Validation auc = 0.681221
Epoch 151
Validation auc = 0.679343
Epoch 152
Validation auc = 0.684038
Epoch 153
Validation auc = 0.685446
Epoch 154
Loss = 2.4529e-01, PNorm = 23.4257, GNorm = 2.0711, lr_0 = 7.7759e-04
Validation auc = 0.681690
Epoch 155
Validation auc = 0.683568
Epoch 156
Validation auc = 0.679812
Epoch 157
Validation auc = 0.680282
Epoch 158
Validation auc = 0.682629
Epoch 159
Loss = 1.6515e-01, PNorm = 23.5059, GNorm = 1.3240, lr_0 = 7.7095e-04
Validation auc = 0.683568
Epoch 160
Validation auc = 0.684507
Epoch 161
Validation auc = 0.688732
Epoch 162
Validation auc = 0.680282
Epoch 163
Validation auc = 0.684507
Epoch 164
Loss = 1.4884e-01, PNorm = 23.5733, GNorm = 1.1152, lr_0 = 7.6436e-04
Validation auc = 0.682160
Epoch 165
Validation auc = 0.685915
Epoch 166
Validation auc = 0.686385
Epoch 167
Validation auc = 0.687793
Epoch 168
Validation auc = 0.684038
Epoch 169
Loss = 8.7490e-02, PNorm = 23.6342, GNorm = 1.3684, lr_0 = 7.5782e-04
Validation auc = 0.685915
Epoch 170
Validation auc = 0.685915
Epoch 171
Validation auc = 0.685915
Epoch 172
Validation auc = 0.688263
Epoch 173
Validation auc = 0.684507
Epoch 174
Loss = 2.2728e-01, PNorm = 23.6940, GNorm = 3.7848, lr_0 = 7.5134e-04
Validation auc = 0.687324
Epoch 175
Validation auc = 0.686854
Epoch 176
Validation auc = 0.687324
Epoch 177
Validation auc = 0.689671
Epoch 178
Validation auc = 0.688263
Epoch 179
Loss = 1.7715e-01, PNorm = 23.7622, GNorm = 1.1449, lr_0 = 7.4492e-04
Validation auc = 0.689202
Epoch 180
Validation auc = 0.686854
Epoch 181
Validation auc = 0.684507
Epoch 182
Validation auc = 0.686385
Epoch 183
Validation auc = 0.681221
Epoch 184
Loss = 1.1600e-01, PNorm = 23.8385, GNorm = 1.4442, lr_0 = 7.3855e-04
Validation auc = 0.679812
Epoch 185
Validation auc = 0.684507
Epoch 186
Validation auc = 0.684038
Epoch 187
Validation auc = 0.684507
Epoch 188
Validation auc = 0.684038
Epoch 189
Loss = 1.5698e-01, PNorm = 23.9087, GNorm = 0.9594, lr_0 = 7.3224e-04
Validation auc = 0.680282
Epoch 190
Validation auc = 0.674178
Epoch 191
Validation auc = 0.670423
Epoch 192
Validation auc = 0.669484
Epoch 193
Validation auc = 0.673709
Epoch 194
Loss = 1.1894e-01, PNorm = 23.9679, GNorm = 1.0103, lr_0 = 7.2598e-04
Validation auc = 0.674178
Epoch 195
Validation auc = 0.675587
Epoch 196
Validation auc = 0.670892
Epoch 197
Validation auc = 0.669014
Epoch 198
Validation auc = 0.666667
Epoch 199
Loss = 1.5722e-01, PNorm = 24.0274, GNorm = 1.3005, lr_0 = 7.1977e-04
Validation auc = 0.664789
Epoch 200
Validation auc = 0.667136
Epoch 201
Validation auc = 0.673239
Epoch 202
Validation auc = 0.675587
Epoch 203
Validation auc = 0.678873
Epoch 204
Loss = 1.5217e-01, PNorm = 24.0870, GNorm = 1.3633, lr_0 = 7.1362e-04
Validation auc = 0.671362
Epoch 205
Validation auc = 0.669953
Epoch 206
Validation auc = 0.663850
Epoch 207
Validation auc = 0.654460
Epoch 208
Validation auc = 0.649296
Epoch 209
Loss = 1.5562e-01, PNorm = 24.1443, GNorm = 2.2195, lr_0 = 7.0752e-04
Validation auc = 0.646948
Epoch 210
Validation auc = 0.650704
Epoch 211
Validation auc = 0.653052
Epoch 212
Validation auc = 0.657746
Epoch 213
Validation auc = 0.662911
Epoch 214
Loss = 1.5857e-01, PNorm = 24.2065, GNorm = 1.2730, lr_0 = 7.0147e-04
Validation auc = 0.662911
Epoch 215
Validation auc = 0.674648
Epoch 216
Validation auc = 0.676056
Epoch 217
Validation auc = 0.677934
Epoch 218
Validation auc = 0.671362
Epoch 219
Loss = 1.6871e-01, PNorm = 24.2702, GNorm = 3.8023, lr_0 = 6.9548e-04
Validation auc = 0.669014
Epoch 220
Validation auc = 0.661033
Epoch 221
Validation auc = 0.663850
Epoch 222
Validation auc = 0.657746
Epoch 223
Validation auc = 0.650704
Epoch 224
Loss = 1.4515e-01, PNorm = 24.3394, GNorm = 0.8835, lr_0 = 6.8953e-04
Validation auc = 0.656338
Epoch 225
Validation auc = 0.661502
Epoch 226
Validation auc = 0.659624
Epoch 227
Validation auc = 0.658216
Epoch 228
Validation auc = 0.662441
Epoch 229
Loss = 1.8453e-01, PNorm = 24.4038, GNorm = 2.6704, lr_0 = 6.8364e-04
Validation auc = 0.665728
Epoch 230
Validation auc = 0.669484
Epoch 231
Validation auc = 0.667136
Epoch 232
Validation auc = 0.664789
Epoch 233
Validation auc = 0.656808
Epoch 234
Loss = 2.0570e-01, PNorm = 24.4665, GNorm = 3.4133, lr_0 = 6.7779e-04
Validation auc = 0.656808
Epoch 235
Validation auc = 0.663850
Epoch 236
Validation auc = 0.667136
Epoch 237
Validation auc = 0.671362
Epoch 238
Validation auc = 0.673709
Epoch 239
Loss = 1.2379e-01, PNorm = 24.5195, GNorm = 1.6189, lr_0 = 6.7200e-04
Validation auc = 0.673709
Epoch 240
Validation auc = 0.674178
Epoch 241
Validation auc = 0.674178
Epoch 242
Validation auc = 0.674648
Epoch 243
Validation auc = 0.667606
Epoch 244
Loss = 2.0954e-01, PNorm = 24.5781, GNorm = 4.4390, lr_0 = 6.6625e-04
Validation auc = 0.670423
Epoch 245
Validation auc = 0.682629
Epoch 246
Validation auc = 0.687324
Epoch 247
Validation auc = 0.684977
Epoch 248
Validation auc = 0.684977
Epoch 249
Loss = 1.5128e-01, PNorm = 24.6461, GNorm = 2.7131, lr_0 = 6.6056e-04
Validation auc = 0.679812
Epoch 250
Validation auc = 0.678873
Epoch 251
Validation auc = 0.683099
Epoch 252
Validation auc = 0.682629
Epoch 253
Validation auc = 0.680282
Epoch 254
Loss = 1.4841e-01, PNorm = 24.7096, GNorm = 1.1822, lr_0 = 6.5491e-04
Validation auc = 0.681221
Epoch 255
Validation auc = 0.673709
Epoch 256
Validation auc = 0.672770
Epoch 257
Validation auc = 0.673239
Epoch 258
Validation auc = 0.674648
Epoch 259
Loss = 2.5479e-01, PNorm = 24.7722, GNorm = 1.9334, lr_0 = 6.4931e-04
Validation auc = 0.675587
Epoch 260
Validation auc = 0.671362
Epoch 261
Validation auc = 0.668075
Epoch 262
Validation auc = 0.669953
Epoch 263
Validation auc = 0.670892
Epoch 264
Loss = 1.0236e-01, PNorm = 24.8202, GNorm = 3.0441, lr_0 = 6.4376e-04
Validation auc = 0.671831
Epoch 265
Validation auc = 0.673709
Epoch 266
Validation auc = 0.673239
Epoch 267
Validation auc = 0.669014
Epoch 268
Validation auc = 0.667606
Epoch 269
Loss = 8.0855e-02, PNorm = 24.8768, GNorm = 1.7100, lr_0 = 6.3826e-04
Validation auc = 0.664789
Epoch 270
Validation auc = 0.661033
Epoch 271
Validation auc = 0.656338
Epoch 272
Validation auc = 0.650704
Epoch 273
Validation auc = 0.657277
Epoch 274
Loss = 1.1614e-01, PNorm = 24.9312, GNorm = 1.0513, lr_0 = 6.3280e-04
Validation auc = 0.663380
Epoch 275
Validation auc = 0.666197
Epoch 276
Validation auc = 0.667606
Epoch 277
Validation auc = 0.667136
Epoch 278
Validation auc = 0.669484
Epoch 279
Loss = 1.1051e-01, PNorm = 24.9823, GNorm = 1.6558, lr_0 = 6.2739e-04
Validation auc = 0.672770
Epoch 280
Validation auc = 0.670423
Epoch 281
Validation auc = 0.669014
Epoch 282
Validation auc = 0.664319
Epoch 283
Validation auc = 0.654460
Epoch 284
Loss = 2.8073e-01, PNorm = 25.0323, GNorm = 5.4285, lr_0 = 6.2203e-04
Validation auc = 0.653991
Epoch 285
Validation auc = 0.672300
Epoch 286
Validation auc = 0.674648
Epoch 287
Validation auc = 0.674648
Epoch 288
Validation auc = 0.674178
Epoch 289
Loss = 1.1832e-01, PNorm = 25.0881, GNorm = 1.6410, lr_0 = 6.1671e-04
Validation auc = 0.672770
Epoch 290
Validation auc = 0.668545
Epoch 291
Validation auc = 0.662441
Epoch 292
Validation auc = 0.657277
Epoch 293
Validation auc = 0.644131
Epoch 294
Loss = 1.2836e-01, PNorm = 25.1406, GNorm = 1.5429, lr_0 = 6.1144e-04
Validation auc = 0.630986
Epoch 295
Validation auc = 0.615962
Epoch 296
Validation auc = 0.619249
Epoch 297
Validation auc = 0.647418
Epoch 298
Validation auc = 0.663380
Epoch 299
Loss = 1.4747e-01, PNorm = 25.1918, GNorm = 3.2764, lr_0 = 6.0621e-04
Validation auc = 0.672300
Model 0 best validation auc = 0.730047 on epoch 32
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.886213
Ensemble test auc = 0.886213
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.515962
Epoch 1
Validation auc = 0.597653
Epoch 2
Validation auc = 0.622535
Epoch 3
Validation auc = 0.645070
Epoch 4
Loss = 7.6149e-01, PNorm = 20.9843, GNorm = 2.8595, lr_0 = 6.5000e-04
Validation auc = 0.672770
Epoch 5
Validation auc = 0.662441
Epoch 6
Validation auc = 0.624883
Epoch 7
Validation auc = 0.642723
Epoch 8
Validation auc = 0.639906
Epoch 9
Loss = 5.7650e-01, PNorm = 21.0291, GNorm = 5.1069, lr_0 = 9.9743e-04
Validation auc = 0.622535
Epoch 10
Validation auc = 0.649765
Epoch 11
Validation auc = 0.645540
Epoch 12
Validation auc = 0.625822
Epoch 13
Validation auc = 0.624413
Epoch 14
Loss = 5.8389e-01, PNorm = 21.0972, GNorm = 2.1914, lr_0 = 9.8890e-04
Validation auc = 0.654930
Epoch 15
Validation auc = 0.676526
Epoch 16
Validation auc = 0.682160
Epoch 17
Validation auc = 0.683568
Epoch 18
Validation auc = 0.679343
Epoch 19
Loss = 5.3144e-01, PNorm = 21.1577, GNorm = 5.5018, lr_0 = 9.8045e-04
Validation auc = 0.674648
Epoch 20
Validation auc = 0.681221
Epoch 21
Validation auc = 0.682160
Epoch 22
Validation auc = 0.685446
Epoch 23
Validation auc = 0.676526
Epoch 24
Loss = 4.5585e-01, PNorm = 21.2217, GNorm = 2.3089, lr_0 = 9.7207e-04
Validation auc = 0.679812
Epoch 25
Validation auc = 0.684507
Epoch 26
Validation auc = 0.684977
Epoch 27
Validation auc = 0.687793
Epoch 28
Validation auc = 0.687324
Epoch 29
Loss = 3.5422e-01, PNorm = 21.2855, GNorm = 4.6816, lr_0 = 9.6376e-04
Validation auc = 0.695305
Epoch 30
Validation auc = 0.687324
Epoch 31
Validation auc = 0.692958
Epoch 32
Validation auc = 0.700939
Epoch 33
Validation auc = 0.704695
Epoch 34
Loss = 3.2388e-01, PNorm = 21.3541, GNorm = 2.3252, lr_0 = 9.5552e-04
Validation auc = 0.692019
Epoch 35
Validation auc = 0.676995
Epoch 36
Validation auc = 0.678873
Epoch 37
Validation auc = 0.676995
Epoch 38
Validation auc = 0.698122
Epoch 39
Loss = 3.7956e-01, PNorm = 21.4265, GNorm = 4.1875, lr_0 = 9.4735e-04
Validation auc = 0.703286
Epoch 40
Validation auc = 0.707512
Epoch 41
Validation auc = 0.703756
Epoch 42
Validation auc = 0.695305
Epoch 43
Validation auc = 0.694366
Epoch 44
Loss = 3.3977e-01, PNorm = 21.5082, GNorm = 3.7921, lr_0 = 9.3925e-04
Validation auc = 0.690141
Epoch 45
Validation auc = 0.691549
Epoch 46
Validation auc = 0.693427
Epoch 47
Validation auc = 0.700469
Epoch 48
Validation auc = 0.701878
Epoch 49
Loss = 4.4013e-01, PNorm = 21.5887, GNorm = 1.6683, lr_0 = 9.3122e-04
Validation auc = 0.700000
Epoch 50
Validation auc = 0.704225
Epoch 51
Validation auc = 0.706103
Epoch 52
Validation auc = 0.706103
Epoch 53
Validation auc = 0.706103
Epoch 54
Loss = 3.2943e-01, PNorm = 21.6819, GNorm = 2.7172, lr_0 = 9.2326e-04
Validation auc = 0.703286
Epoch 55
Validation auc = 0.703756
Epoch 56
Validation auc = 0.706103
Epoch 57
Validation auc = 0.701878
Epoch 58
Validation auc = 0.689671
Epoch 59
Loss = 4.0524e-01, PNorm = 21.7607, GNorm = 4.2328, lr_0 = 9.1537e-04
Validation auc = 0.685915
Epoch 60
Validation auc = 0.689671
Epoch 61
Validation auc = 0.689671
Epoch 62
Validation auc = 0.686854
Epoch 63
Validation auc = 0.684507
Epoch 64
Loss = 2.4438e-01, PNorm = 21.8464, GNorm = 1.8671, lr_0 = 9.0754e-04
Validation auc = 0.679812
Epoch 65
Validation auc = 0.677934
Epoch 66
Validation auc = 0.670892
Epoch 67
Validation auc = 0.676526
Epoch 68
Validation auc = 0.671831
Epoch 69
Loss = 2.5522e-01, PNorm = 21.9361, GNorm = 2.3944, lr_0 = 8.9978e-04
Validation auc = 0.661502
Epoch 70
Validation auc = 0.685446
Epoch 71
Validation auc = 0.698592
Epoch 72
Validation auc = 0.700939
Epoch 73
Validation auc = 0.702817
Epoch 74
Loss = 3.9355e-01, PNorm = 22.0299, GNorm = 7.3871, lr_0 = 8.9209e-04
Validation auc = 0.709390
Epoch 75
Validation auc = 0.713615
Epoch 76
Validation auc = 0.706103
Epoch 77
Validation auc = 0.706103
Epoch 78
Validation auc = 0.699531
Epoch 79
Loss = 2.6936e-01, PNorm = 22.1333, GNorm = 2.1527, lr_0 = 8.8447e-04
Validation auc = 0.706103
Epoch 80
Validation auc = 0.707042
Epoch 81
Validation auc = 0.701878
Epoch 82
Validation auc = 0.696244
Epoch 83
Validation auc = 0.699061
Epoch 84
Loss = 3.0706e-01, PNorm = 22.2166, GNorm = 1.9594, lr_0 = 8.7691e-04
Validation auc = 0.700939
Epoch 85
Validation auc = 0.702347
Epoch 86
Validation auc = 0.698592
Epoch 87
Validation auc = 0.695775
Epoch 88
Validation auc = 0.694836
Epoch 89
Loss = 2.1575e-01, PNorm = 22.2957, GNorm = 1.5189, lr_0 = 8.6941e-04
Validation auc = 0.691080
Epoch 90
Validation auc = 0.692488
Epoch 91
Validation auc = 0.684977
Epoch 92
Validation auc = 0.672770
Epoch 93
Validation auc = 0.673239
Epoch 94
Loss = 3.3070e-01, PNorm = 22.3810, GNorm = 2.5390, lr_0 = 8.6198e-04
Validation auc = 0.680751
Epoch 95
Validation auc = 0.676995
Epoch 96
Validation auc = 0.674648
Epoch 97
Validation auc = 0.672300
Epoch 98
Validation auc = 0.683568
Epoch 99
Loss = 1.9521e-01, PNorm = 22.4575, GNorm = 2.5973, lr_0 = 8.5461e-04
Validation auc = 0.691549
Epoch 100
Validation auc = 0.690610
Epoch 101
Validation auc = 0.684977
Epoch 102
Validation auc = 0.690141
Epoch 103
Validation auc = 0.690141
Epoch 104
Loss = 2.1921e-01, PNorm = 22.5274, GNorm = 4.8877, lr_0 = 8.4730e-04
Validation auc = 0.691549
Epoch 105
Validation auc = 0.688263
Epoch 106
Validation auc = 0.689671
Epoch 107
Validation auc = 0.693427
Epoch 108
Validation auc = 0.698592
Epoch 109
Loss = 2.9847e-01, PNorm = 22.6083, GNorm = 1.6408, lr_0 = 8.4006e-04
Validation auc = 0.698122
Epoch 110
Validation auc = 0.703756
Epoch 111
Validation auc = 0.701408
Epoch 112
Validation auc = 0.699061
Epoch 113
Validation auc = 0.685915
Epoch 114
Loss = 2.2123e-01, PNorm = 22.6950, GNorm = 1.6883, lr_0 = 8.3288e-04
Validation auc = 0.662911
Epoch 115
Validation auc = 0.672770
Epoch 116
Validation auc = 0.685915
Epoch 117
Validation auc = 0.683568
Epoch 118
Validation auc = 0.683568
Epoch 119
Loss = 3.3061e-01, PNorm = 22.7902, GNorm = 1.5383, lr_0 = 8.2576e-04
Validation auc = 0.684507
Epoch 120
Validation auc = 0.688263
Epoch 121
Validation auc = 0.689671
Epoch 122
Validation auc = 0.693427
Epoch 123
Validation auc = 0.697653
Epoch 124
Loss = 2.4317e-01, PNorm = 22.8691, GNorm = 1.5973, lr_0 = 8.1870e-04
Validation auc = 0.700000
Epoch 125
Validation auc = 0.698122
Epoch 126
Validation auc = 0.699061
Epoch 127
Validation auc = 0.703756
Epoch 128
Validation auc = 0.703756
Epoch 129
Loss = 1.8872e-01, PNorm = 22.9466, GNorm = 0.8983, lr_0 = 8.1170e-04
Validation auc = 0.699061
Epoch 130
Validation auc = 0.695305
Epoch 131
Validation auc = 0.696244
Epoch 132
Validation auc = 0.690141
Epoch 133
Validation auc = 0.685915
Epoch 134
Loss = 2.0388e-01, PNorm = 23.0201, GNorm = 2.3201, lr_0 = 8.0476e-04
Validation auc = 0.682160
Epoch 135
Validation auc = 0.680282
Epoch 136
Validation auc = 0.683099
Epoch 137
Validation auc = 0.688732
Epoch 138
Validation auc = 0.688732
Epoch 139
Loss = 2.3511e-01, PNorm = 23.0991, GNorm = 1.6269, lr_0 = 7.9788e-04
Validation auc = 0.684038
Epoch 140
Validation auc = 0.680282
Epoch 141
Validation auc = 0.684038
Epoch 142
Validation auc = 0.683568
Epoch 143
Validation auc = 0.686854
Epoch 144
Loss = 1.8522e-01, PNorm = 23.1680, GNorm = 1.9189, lr_0 = 7.9106e-04
Validation auc = 0.685915
Epoch 145
Validation auc = 0.689671
Epoch 146
Validation auc = 0.688263
Epoch 147
Validation auc = 0.684507
Epoch 148
Validation auc = 0.688263
Epoch 149
Loss = 2.2561e-01, PNorm = 23.2442, GNorm = 1.7482, lr_0 = 7.8430e-04
Validation auc = 0.681221
Epoch 150
Validation auc = 0.670892
Epoch 151
Validation auc = 0.666197
Epoch 152
Validation auc = 0.664789
Epoch 153
Validation auc = 0.661033
Epoch 154
Loss = 1.6550e-01, PNorm = 23.3167, GNorm = 2.1854, lr_0 = 7.7759e-04
Validation auc = 0.655399
Epoch 155
Validation auc = 0.653052
Epoch 156
Validation auc = 0.656338
Epoch 157
Validation auc = 0.666667
Epoch 158
Validation auc = 0.672770
Epoch 159
Loss = 2.3105e-01, PNorm = 23.3936, GNorm = 3.9051, lr_0 = 7.7095e-04
Validation auc = 0.676526
Epoch 160
Validation auc = 0.676526
Epoch 161
Validation auc = 0.673239
Epoch 162
Validation auc = 0.676995
Epoch 163
Validation auc = 0.680751
Epoch 164
Loss = 2.3543e-01, PNorm = 23.4717, GNorm = 2.6474, lr_0 = 7.6436e-04
Validation auc = 0.679343
Epoch 165
Validation auc = 0.675117
Epoch 166
Validation auc = 0.674178
Epoch 167
Validation auc = 0.681221
Epoch 168
Validation auc = 0.680282
Epoch 169
Loss = 1.5266e-01, PNorm = 23.5411, GNorm = 1.4669, lr_0 = 7.5782e-04
Validation auc = 0.678873
Epoch 170
Validation auc = 0.671831
Epoch 171
Validation auc = 0.671362
Epoch 172
Validation auc = 0.665728
Epoch 173
Validation auc = 0.662441
Epoch 174
Loss = 1.9872e-01, PNorm = 23.6082, GNorm = 2.2540, lr_0 = 7.5134e-04
Validation auc = 0.660563
Epoch 175
Validation auc = 0.646479
Epoch 176
Validation auc = 0.646948
Epoch 177
Validation auc = 0.657277
Epoch 178
Validation auc = 0.661502
Epoch 179
Loss = 1.8651e-01, PNorm = 23.6695, GNorm = 1.3473, lr_0 = 7.4492e-04
Validation auc = 0.664319
Epoch 180
Validation auc = 0.663850
Epoch 181
Validation auc = 0.661502
Epoch 182
Validation auc = 0.664319
Epoch 183
Validation auc = 0.661502
Epoch 184
Loss = 1.9923e-01, PNorm = 23.7324, GNorm = 1.6066, lr_0 = 7.3855e-04
Validation auc = 0.659155
Epoch 185
Validation auc = 0.661033
Epoch 186
Validation auc = 0.662441
Epoch 187
Validation auc = 0.664789
Epoch 188
Validation auc = 0.668075
Epoch 189
Loss = 1.2523e-01, PNorm = 23.7930, GNorm = 1.5449, lr_0 = 7.3224e-04
Validation auc = 0.670423
Epoch 190
Validation auc = 0.673239
Epoch 191
Validation auc = 0.672770
Epoch 192
Validation auc = 0.671362
Epoch 193
Validation auc = 0.666667
Epoch 194
Loss = 2.2286e-01, PNorm = 23.8532, GNorm = 1.7802, lr_0 = 7.2598e-04
Validation auc = 0.666667
Epoch 195
Validation auc = 0.661033
Epoch 196
Validation auc = 0.660094
Epoch 197
Validation auc = 0.662441
Epoch 198
Validation auc = 0.662911
Epoch 199
Loss = 1.3442e-01, PNorm = 23.9157, GNorm = 1.2230, lr_0 = 7.1977e-04
Validation auc = 0.669484
Epoch 200
Validation auc = 0.670423
Epoch 201
Validation auc = 0.673239
Epoch 202
Validation auc = 0.676526
Epoch 203
Validation auc = 0.680751
Epoch 204
Loss = 1.6719e-01, PNorm = 23.9736, GNorm = 2.4133, lr_0 = 7.1362e-04
Validation auc = 0.676995
Epoch 205
Validation auc = 0.673709
Epoch 206
Validation auc = 0.671362
Epoch 207
Validation auc = 0.682629
Epoch 208
Validation auc = 0.689202
Epoch 209
Loss = 2.5035e-01, PNorm = 24.0521, GNorm = 4.6855, lr_0 = 7.0752e-04
Validation auc = 0.683099
Epoch 210
Validation auc = 0.672770
Epoch 211
Validation auc = 0.674178
Epoch 212
Validation auc = 0.679343
Epoch 213
Validation auc = 0.681221
Epoch 214
Loss = 1.8351e-01, PNorm = 24.1375, GNorm = 1.3011, lr_0 = 7.0147e-04
Validation auc = 0.683568
Epoch 215
Validation auc = 0.680282
Epoch 216
Validation auc = 0.675117
Epoch 217
Validation auc = 0.676056
Epoch 218
Validation auc = 0.681690
Epoch 219
Loss = 2.0484e-01, PNorm = 24.2066, GNorm = 2.1221, lr_0 = 6.9548e-04
Validation auc = 0.683568
Epoch 220
Validation auc = 0.678404
Epoch 221
Validation auc = 0.671362
Epoch 222
Validation auc = 0.676056
Epoch 223
Validation auc = 0.683568
Epoch 224
Loss = 1.5564e-01, PNorm = 24.2677, GNorm = 1.6668, lr_0 = 6.8953e-04
Validation auc = 0.684507
Epoch 225
Validation auc = 0.676056
Epoch 226
Validation auc = 0.669953
Epoch 227
Validation auc = 0.668075
Epoch 228
Validation auc = 0.666667
Epoch 229
Loss = 1.3775e-01, PNorm = 24.3249, GNorm = 1.5150, lr_0 = 6.8364e-04
Validation auc = 0.672300
Epoch 230
Validation auc = 0.674178
Epoch 231
Validation auc = 0.673709
Epoch 232
Validation auc = 0.674648
Epoch 233
Validation auc = 0.674648
Epoch 234
Loss = 1.6480e-01, PNorm = 24.3840, GNorm = 1.6501, lr_0 = 6.7779e-04
Validation auc = 0.676056
Epoch 235
Validation auc = 0.675117
Epoch 236
Validation auc = 0.660563
Epoch 237
Validation auc = 0.647418
Epoch 238
Validation auc = 0.654460
Epoch 239
Loss = 1.3422e-01, PNorm = 24.4483, GNorm = 3.0973, lr_0 = 6.7200e-04
Validation auc = 0.676056
Epoch 240
Validation auc = 0.685915
Epoch 241
Validation auc = 0.695775
Epoch 242
Validation auc = 0.699061
Epoch 243
Validation auc = 0.703756
Epoch 244
Loss = 2.0771e-01, PNorm = 24.5151, GNorm = 1.9395, lr_0 = 6.6625e-04
Validation auc = 0.703756
Epoch 245
Validation auc = 0.700939
Epoch 246
Validation auc = 0.700939
Epoch 247
Validation auc = 0.698592
Epoch 248
Validation auc = 0.696714
Epoch 249
Loss = 2.0254e-01, PNorm = 24.5731, GNorm = 4.5935, lr_0 = 6.6056e-04
Validation auc = 0.693897
Epoch 250
Validation auc = 0.695305
Epoch 251
Validation auc = 0.693897
Epoch 252
Validation auc = 0.701408
Epoch 253
Validation auc = 0.702347
Epoch 254
Loss = 1.8168e-01, PNorm = 24.6309, GNorm = 2.3895, lr_0 = 6.5491e-04
Validation auc = 0.700000
Epoch 255
Validation auc = 0.692488
Epoch 256
Validation auc = 0.692958
Epoch 257
Validation auc = 0.690610
Epoch 258
Validation auc = 0.691080
Epoch 259
Loss = 1.3749e-01, PNorm = 24.6881, GNorm = 1.5671, lr_0 = 6.4931e-04
Validation auc = 0.691549
Epoch 260
Validation auc = 0.691549
Epoch 261
Validation auc = 0.694366
Epoch 262
Validation auc = 0.693427
Epoch 263
Validation auc = 0.689202
Epoch 264
Loss = 8.6017e-02, PNorm = 24.7479, GNorm = 1.4107, lr_0 = 6.4376e-04
Validation auc = 0.685915
Epoch 265
Validation auc = 0.688732
Epoch 266
Validation auc = 0.686854
Epoch 267
Validation auc = 0.687793
Epoch 268
Validation auc = 0.684507
Epoch 269
Loss = 1.6633e-01, PNorm = 24.7995, GNorm = 2.1275, lr_0 = 6.3826e-04
Validation auc = 0.684977
Epoch 270
Validation auc = 0.684507
Epoch 271
Validation auc = 0.684507
Epoch 272
Validation auc = 0.685446
Epoch 273
Validation auc = 0.687793
Epoch 274
Loss = 9.5619e-02, PNorm = 24.8489, GNorm = 1.0642, lr_0 = 6.3280e-04
Validation auc = 0.684977
Epoch 275
Validation auc = 0.689202
Epoch 276
Validation auc = 0.692958
Epoch 277
Validation auc = 0.693897
Epoch 278
Validation auc = 0.692488
Epoch 279
Loss = 1.5068e-01, PNorm = 24.9099, GNorm = 1.1935, lr_0 = 6.2739e-04
Validation auc = 0.700469
Epoch 280
Validation auc = 0.698122
Epoch 281
Validation auc = 0.695305
Epoch 282
Validation auc = 0.694366
Epoch 283
Validation auc = 0.694836
Epoch 284
Loss = 1.6138e-01, PNorm = 24.9728, GNorm = 2.1554, lr_0 = 6.2203e-04
Validation auc = 0.696714
Epoch 285
Validation auc = 0.692958
Epoch 286
Validation auc = 0.690610
Epoch 287
Validation auc = 0.685446
Epoch 288
Validation auc = 0.690141
Epoch 289
Loss = 1.4866e-01, PNorm = 25.0341, GNorm = 3.8202, lr_0 = 6.1671e-04
Validation auc = 0.687793
Epoch 290
Validation auc = 0.694836
Epoch 291
Validation auc = 0.701408
Epoch 292
Validation auc = 0.705164
Epoch 293
Validation auc = 0.708920
Epoch 294
Loss = 1.1990e-01, PNorm = 25.0969, GNorm = 2.7018, lr_0 = 6.1144e-04
Validation auc = 0.707981
Epoch 295
Validation auc = 0.708920
Epoch 296
Validation auc = 0.712676
Epoch 297
Validation auc = 0.716432
Epoch 298
Validation auc = 0.710798
Epoch 299
Loss = 8.8045e-02, PNorm = 25.1506, GNorm = 1.3599, lr_0 = 6.0621e-04
Validation auc = 0.708920
Model 0 best validation auc = 0.716432 on epoch 297
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.859121
Ensemble test auc = 0.859121
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.515962
Epoch 1
Validation auc = 0.601408
Epoch 2
Validation auc = 0.654930
Epoch 3
Validation auc = 0.675587
Epoch 4
Loss = 7.0262e-01, PNorm = 20.9835, GNorm = 3.8418, lr_0 = 6.5000e-04
Validation auc = 0.645540
Epoch 5
Validation auc = 0.647418
Epoch 6
Validation auc = 0.619249
Epoch 7
Validation auc = 0.648826
Epoch 8
Validation auc = 0.691549
Epoch 9
Loss = 5.8118e-01, PNorm = 21.0280, GNorm = 4.8817, lr_0 = 9.9743e-04
Validation auc = 0.689671
Epoch 10
Validation auc = 0.705164
Epoch 11
Validation auc = 0.716901
Epoch 12
Validation auc = 0.725352
Epoch 13
Validation auc = 0.729577
Epoch 14
Loss = 5.3373e-01, PNorm = 21.0949, GNorm = 6.1391, lr_0 = 9.8890e-04
Validation auc = 0.722066
Epoch 15
Validation auc = 0.718779
Epoch 16
Validation auc = 0.706573
Epoch 17
Validation auc = 0.710798
Epoch 18
Validation auc = 0.708451
Epoch 19
Loss = 4.7991e-01, PNorm = 21.1590, GNorm = 3.4481, lr_0 = 9.8045e-04
Validation auc = 0.715962
Epoch 20
Validation auc = 0.723944
Epoch 21
Validation auc = 0.728169
Epoch 22
Validation auc = 0.735681
Epoch 23
Validation auc = 0.750704
Epoch 24
Loss = 5.4603e-01, PNorm = 21.2230, GNorm = 8.4707, lr_0 = 9.7207e-04
Validation auc = 0.738967
Epoch 25
Validation auc = 0.731455
Epoch 26
Validation auc = 0.715962
Epoch 27
Validation auc = 0.697653
Epoch 28
Validation auc = 0.696714
Epoch 29
Loss = 3.7576e-01, PNorm = 21.2919, GNorm = 3.1365, lr_0 = 9.6376e-04
Validation auc = 0.698592
Epoch 30
Validation auc = 0.695775
Epoch 31
Validation auc = 0.695305
Epoch 32
Validation auc = 0.688732
Epoch 33
Validation auc = 0.697183
Epoch 34
Loss = 3.8086e-01, PNorm = 21.3570, GNorm = 8.0466, lr_0 = 9.5552e-04
Validation auc = 0.690610
Epoch 35
Validation auc = 0.691549
Epoch 36
Validation auc = 0.690610
Epoch 37
Validation auc = 0.695305
Epoch 38
Validation auc = 0.697653
Epoch 39
Loss = 4.5089e-01, PNorm = 21.4230, GNorm = 2.3637, lr_0 = 9.4735e-04
Validation auc = 0.698592
Epoch 40
Validation auc = 0.696244
Epoch 41
Validation auc = 0.697183
Epoch 42
Validation auc = 0.696244
Epoch 43
Validation auc = 0.698122
Epoch 44
Loss = 4.0526e-01, PNorm = 21.4923, GNorm = 1.8165, lr_0 = 9.3925e-04
Validation auc = 0.700469
Epoch 45
Validation auc = 0.702817
Epoch 46
Validation auc = 0.702817
Epoch 47
Validation auc = 0.694366
Epoch 48
Validation auc = 0.685915
Epoch 49
Loss = 3.0045e-01, PNorm = 21.5585, GNorm = 3.6220, lr_0 = 9.3122e-04
Validation auc = 0.687324
Epoch 50
Validation auc = 0.688732
Epoch 51
Validation auc = 0.690141
Epoch 52
Validation auc = 0.691080
Epoch 53
Validation auc = 0.679343
Epoch 54
Loss = 3.1473e-01, PNorm = 21.6366, GNorm = 2.7153, lr_0 = 9.2326e-04
Validation auc = 0.689202
Epoch 55
Validation auc = 0.689202
Epoch 56
Validation auc = 0.691549
Epoch 57
Validation auc = 0.695775
Epoch 58
Validation auc = 0.687793
Epoch 59
Loss = 2.9808e-01, PNorm = 21.7252, GNorm = 1.2400, lr_0 = 9.1537e-04
Validation auc = 0.685915
Epoch 60
Validation auc = 0.678404
Epoch 61
Validation auc = 0.671831
Epoch 62
Validation auc = 0.674178
Epoch 63
Validation auc = 0.676995
Epoch 64
Loss = 2.5813e-01, PNorm = 21.8129, GNorm = 1.7044, lr_0 = 9.0754e-04
Validation auc = 0.664789
Epoch 65
Validation auc = 0.672770
Epoch 66
Validation auc = 0.681221
Epoch 67
Validation auc = 0.679343
Epoch 68
Validation auc = 0.684977
Epoch 69
Loss = 3.1945e-01, PNorm = 21.8951, GNorm = 6.2434, lr_0 = 8.9978e-04
Validation auc = 0.685446
Epoch 70
Validation auc = 0.685446
Epoch 71
Validation auc = 0.691080
Epoch 72
Validation auc = 0.680282
Epoch 73
Validation auc = 0.686385
Epoch 74
Loss = 3.0779e-01, PNorm = 21.9832, GNorm = 2.3521, lr_0 = 8.9209e-04
Validation auc = 0.684038
Epoch 75
Validation auc = 0.677934
Epoch 76
Validation auc = 0.668075
Epoch 77
Validation auc = 0.674648
Epoch 78
Validation auc = 0.675587
Epoch 79
Loss = 1.7801e-01, PNorm = 22.0666, GNorm = 3.8336, lr_0 = 8.8447e-04
Validation auc = 0.671362
Epoch 80
Validation auc = 0.673239
Epoch 81
Validation auc = 0.679812
Epoch 82
Validation auc = 0.680282
Epoch 83
Validation auc = 0.683568
Epoch 84
Loss = 2.6734e-01, PNorm = 22.1514, GNorm = 1.5799, lr_0 = 8.7691e-04
Validation auc = 0.688263
Epoch 85
Validation auc = 0.691549
Epoch 86
Validation auc = 0.694836
Epoch 87
Validation auc = 0.695775
Epoch 88
Validation auc = 0.697183
Epoch 89
Loss = 2.3981e-01, PNorm = 22.2450, GNorm = 2.0224, lr_0 = 8.6941e-04
Validation auc = 0.692019
Epoch 90
Validation auc = 0.693427
Epoch 91
Validation auc = 0.693897
Epoch 92
Validation auc = 0.690141
Epoch 93
Validation auc = 0.687793
Epoch 94
Loss = 3.4029e-01, PNorm = 22.3352, GNorm = 2.2943, lr_0 = 8.6198e-04
Validation auc = 0.690141
Epoch 95
Validation auc = 0.691080
Epoch 96
Validation auc = 0.683099
Epoch 97
Validation auc = 0.682629
Epoch 98
Validation auc = 0.677934
Epoch 99
Loss = 2.7538e-01, PNorm = 22.4180, GNorm = 5.9788, lr_0 = 8.5461e-04
Validation auc = 0.670892
Epoch 100
Validation auc = 0.667606
Epoch 101
Validation auc = 0.674178
Epoch 102
Validation auc = 0.678404
Epoch 103
Validation auc = 0.677465
Epoch 104
Loss = 2.8515e-01, PNorm = 22.5082, GNorm = 3.8172, lr_0 = 8.4730e-04
Validation auc = 0.677934
Epoch 105
Validation auc = 0.682629
Epoch 106
Validation auc = 0.683568
Epoch 107
Validation auc = 0.666667
Epoch 108
Validation auc = 0.646479
Epoch 109
Loss = 2.1951e-01, PNorm = 22.5886, GNorm = 3.2444, lr_0 = 8.4006e-04
Validation auc = 0.651174
Epoch 110
Validation auc = 0.672300
Epoch 111
Validation auc = 0.679812
Epoch 112
Validation auc = 0.669953
Epoch 113
Validation auc = 0.674178
Epoch 114
Loss = 2.0890e-01, PNorm = 22.6712, GNorm = 1.6689, lr_0 = 8.3288e-04
Validation auc = 0.669014
Epoch 115
Validation auc = 0.665728
Epoch 116
Validation auc = 0.664319
Epoch 117
Validation auc = 0.658685
Epoch 118
Validation auc = 0.644131
Epoch 119
Loss = 2.1148e-01, PNorm = 22.7501, GNorm = 2.6490, lr_0 = 8.2576e-04
Validation auc = 0.642723
Epoch 120
Validation auc = 0.648357
Epoch 121
Validation auc = 0.638498
Epoch 122
Validation auc = 0.638028
Epoch 123
Validation auc = 0.641315
Epoch 124
Loss = 1.5760e-01, PNorm = 22.8208, GNorm = 1.4440, lr_0 = 8.1870e-04
Validation auc = 0.641315
Epoch 125
Validation auc = 0.645540
Epoch 126
Validation auc = 0.646479
Epoch 127
Validation auc = 0.638498
Epoch 128
Validation auc = 0.636150
Epoch 129
Loss = 1.7404e-01, PNorm = 22.8904, GNorm = 2.3670, lr_0 = 8.1170e-04
Validation auc = 0.643662
Epoch 130
Validation auc = 0.660094
Epoch 131
Validation auc = 0.651174
Epoch 132
Validation auc = 0.637089
Epoch 133
Validation auc = 0.637559
Epoch 134
Loss = 2.2804e-01, PNorm = 22.9715, GNorm = 1.7143, lr_0 = 8.0476e-04
Validation auc = 0.641784
Epoch 135
Validation auc = 0.643662
Epoch 136
Validation auc = 0.644131
Epoch 137
Validation auc = 0.644601
Epoch 138
Validation auc = 0.650235
Epoch 139
Loss = 1.9376e-01, PNorm = 23.0514, GNorm = 2.6544, lr_0 = 7.9788e-04
Validation auc = 0.647887
Epoch 140
Validation auc = 0.652113
Epoch 141
Validation auc = 0.655399
Epoch 142
Validation auc = 0.659155
Epoch 143
Validation auc = 0.669953
Epoch 144
Loss = 1.9858e-01, PNorm = 23.1237, GNorm = 0.8123, lr_0 = 7.9106e-04
Validation auc = 0.674178
Epoch 145
Validation auc = 0.666667
Epoch 146
Validation auc = 0.660094
Epoch 147
Validation auc = 0.664319
Epoch 148
Validation auc = 0.662911
Epoch 149
Loss = 2.5319e-01, PNorm = 23.1948, GNorm = 2.3512, lr_0 = 7.8430e-04
Validation auc = 0.664789
Epoch 150
Validation auc = 0.661033
Epoch 151
Validation auc = 0.676995
Epoch 152
Validation auc = 0.680282
Epoch 153
Validation auc = 0.677934
Epoch 154
Loss = 1.4355e-01, PNorm = 23.2682, GNorm = 2.2792, lr_0 = 7.7759e-04
Validation auc = 0.676995
Epoch 155
Validation auc = 0.672770
Epoch 156
Validation auc = 0.673239
Epoch 157
Validation auc = 0.673239
Epoch 158
Validation auc = 0.673239
Epoch 159
Loss = 1.2304e-01, PNorm = 23.3429, GNorm = 3.5440, lr_0 = 7.7095e-04
Validation auc = 0.670892
Epoch 160
Validation auc = 0.671831
Epoch 161
Validation auc = 0.675587
Epoch 162
Validation auc = 0.676526
Epoch 163
Validation auc = 0.667606
Epoch 164
Loss = 2.6056e-01, PNorm = 23.4170, GNorm = 3.8194, lr_0 = 7.6436e-04
Validation auc = 0.665728
Epoch 165
Validation auc = 0.669953
Epoch 166
Validation auc = 0.673709
Epoch 167
Validation auc = 0.668075
Epoch 168
Validation auc = 0.656808
Epoch 169
Loss = 1.6501e-01, PNorm = 23.4879, GNorm = 1.7703, lr_0 = 7.5782e-04
Validation auc = 0.651643
Epoch 170
Validation auc = 0.666667
Epoch 171
Validation auc = 0.670423
Epoch 172
Validation auc = 0.674648
Epoch 173
Validation auc = 0.669953
Epoch 174
Loss = 2.2412e-01, PNorm = 23.5537, GNorm = 1.6886, lr_0 = 7.5134e-04
Validation auc = 0.672770
Epoch 175
Validation auc = 0.671831
Epoch 176
Validation auc = 0.670423
Epoch 177
Validation auc = 0.674178
Epoch 178
Validation auc = 0.672300
Epoch 179
Loss = 2.1362e-01, PNorm = 23.6194, GNorm = 2.1110, lr_0 = 7.4492e-04
Validation auc = 0.672770
Epoch 180
Validation auc = 0.670423
Epoch 181
Validation auc = 0.677934
Epoch 182
Validation auc = 0.676056
Epoch 183
Validation auc = 0.670892
Epoch 184
Loss = 2.1399e-01, PNorm = 23.6885, GNorm = 1.6566, lr_0 = 7.3855e-04
Validation auc = 0.665258
Epoch 185
Validation auc = 0.661033
Epoch 186
Validation auc = 0.660563
Epoch 187
Validation auc = 0.660094
Epoch 188
Validation auc = 0.662911
Epoch 189
Loss = 1.5638e-01, PNorm = 23.7589, GNorm = 1.4382, lr_0 = 7.3224e-04
Validation auc = 0.664319
Epoch 190
Validation auc = 0.648826
Epoch 191
Validation auc = 0.644601
Epoch 192
Validation auc = 0.645070
Epoch 193
Validation auc = 0.646479
Epoch 194
Loss = 2.4601e-01, PNorm = 23.8355, GNorm = 2.0785, lr_0 = 7.2598e-04
Validation auc = 0.652113
Epoch 195
Validation auc = 0.659155
Epoch 196
Validation auc = 0.666197
Epoch 197
Validation auc = 0.673239
Epoch 198
Validation auc = 0.677934
Epoch 199
Loss = 1.5634e-01, PNorm = 23.9047, GNorm = 1.1211, lr_0 = 7.1977e-04
Validation auc = 0.676056
Epoch 200
Validation auc = 0.672770
Epoch 201
Validation auc = 0.669014
Epoch 202
Validation auc = 0.669484
Epoch 203
Validation auc = 0.663850
Epoch 204
Loss = 1.6017e-01, PNorm = 23.9709, GNorm = 1.8671, lr_0 = 7.1362e-04
Validation auc = 0.655869
Epoch 205
Validation auc = 0.651643
Epoch 206
Validation auc = 0.659155
Epoch 207
Validation auc = 0.660094
Epoch 208
Validation auc = 0.649765
Epoch 209
Loss = 1.2384e-01, PNorm = 24.0395, GNorm = 0.9985, lr_0 = 7.0752e-04
Validation auc = 0.640845
Epoch 210
Validation auc = 0.630986
Epoch 211
Validation auc = 0.628169
Epoch 212
Validation auc = 0.631455
Epoch 213
Validation auc = 0.622535
Epoch 214
Loss = 2.3185e-01, PNorm = 24.1121, GNorm = 2.4991, lr_0 = 7.0147e-04
Validation auc = 0.618779
Epoch 215
Validation auc = 0.639437
Epoch 216
Validation auc = 0.665258
Epoch 217
Validation auc = 0.668075
Epoch 218
Validation auc = 0.671362
Epoch 219
Loss = 1.8272e-01, PNorm = 24.1926, GNorm = 1.4169, lr_0 = 6.9548e-04
Validation auc = 0.671831
Epoch 220
Validation auc = 0.669953
Epoch 221
Validation auc = 0.660563
Epoch 222
Validation auc = 0.657746
Epoch 223
Validation auc = 0.653052
Epoch 224
Loss = 1.6952e-01, PNorm = 24.2632, GNorm = 2.1662, lr_0 = 6.8953e-04
Validation auc = 0.657277
Epoch 225
Validation auc = 0.665728
Epoch 226
Validation auc = 0.672770
Epoch 227
Validation auc = 0.676056
Epoch 228
Validation auc = 0.679343
Epoch 229
Loss = 1.7741e-01, PNorm = 24.3245, GNorm = 1.6649, lr_0 = 6.8364e-04
Validation auc = 0.679812
Epoch 230
Validation auc = 0.678873
Epoch 231
Validation auc = 0.674648
Epoch 232
Validation auc = 0.667136
Epoch 233
Validation auc = 0.655399
Epoch 234
Loss = 1.5784e-01, PNorm = 24.3913, GNorm = 1.5927, lr_0 = 6.7779e-04
Validation auc = 0.648357
Epoch 235
Validation auc = 0.641784
Epoch 236
Validation auc = 0.636150
Epoch 237
Validation auc = 0.640376
Epoch 238
Validation auc = 0.653521
Epoch 239
Loss = 1.9617e-01, PNorm = 24.4626, GNorm = 1.7509, lr_0 = 6.7200e-04
Validation auc = 0.659624
Epoch 240
Validation auc = 0.661972
Epoch 241
Validation auc = 0.667136
Epoch 242
Validation auc = 0.666667
Epoch 243
Validation auc = 0.669484
Epoch 244
Loss = 1.9470e-01, PNorm = 24.5254, GNorm = 2.1206, lr_0 = 6.6625e-04
Validation auc = 0.671362
Epoch 245
Validation auc = 0.674178
Epoch 246
Validation auc = 0.671831
Epoch 247
Validation auc = 0.668545
Epoch 248
Validation auc = 0.668075
Epoch 249
Loss = 1.2915e-01, PNorm = 24.5883, GNorm = 2.4635, lr_0 = 6.6056e-04
Validation auc = 0.666667
Epoch 250
Validation auc = 0.669953
Epoch 251
Validation auc = 0.665258
Epoch 252
Validation auc = 0.664789
Epoch 253
Validation auc = 0.663850
Epoch 254
Loss = 1.4573e-01, PNorm = 24.6460, GNorm = 1.8946, lr_0 = 6.5491e-04
Validation auc = 0.661033
Epoch 255
Validation auc = 0.661033
Epoch 256
Validation auc = 0.661502
Epoch 257
Validation auc = 0.661502
Epoch 258
Validation auc = 0.660094
Epoch 259
Loss = 1.4402e-01, PNorm = 24.7022, GNorm = 1.2685, lr_0 = 6.4931e-04
Validation auc = 0.663850
Epoch 260
Validation auc = 0.661972
Epoch 261
Validation auc = 0.661972
Epoch 262
Validation auc = 0.657746
Epoch 263
Validation auc = 0.653991
Epoch 264
Loss = 1.4128e-01, PNorm = 24.7640, GNorm = 1.1730, lr_0 = 6.4376e-04
Validation auc = 0.655869
Epoch 265
Validation auc = 0.652113
Epoch 266
Validation auc = 0.650235
Epoch 267
Validation auc = 0.654460
Epoch 268
Validation auc = 0.661972
Epoch 269
Loss = 1.5960e-01, PNorm = 24.8370, GNorm = 1.4081, lr_0 = 6.3826e-04
Validation auc = 0.664789
Epoch 270
Validation auc = 0.670423
Epoch 271
Validation auc = 0.670423
Epoch 272
Validation auc = 0.673239
Epoch 273
Validation auc = 0.668075
Epoch 274
Loss = 1.6550e-01, PNorm = 24.9054, GNorm = 3.0771, lr_0 = 6.3280e-04
Validation auc = 0.671831
Epoch 275
Validation auc = 0.673239
Epoch 276
Validation auc = 0.681690
Epoch 277
Validation auc = 0.677465
Epoch 278
Validation auc = 0.675117
Epoch 279
Loss = 2.1251e-01, PNorm = 24.9655, GNorm = 4.6120, lr_0 = 6.2739e-04
Validation auc = 0.668075
Epoch 280
Validation auc = 0.663850
Epoch 281
Validation auc = 0.657277
Epoch 282
Validation auc = 0.650235
Epoch 283
Validation auc = 0.654930
Epoch 284
Loss = 2.0334e-01, PNorm = 25.0242, GNorm = 3.5958, lr_0 = 6.2203e-04
Validation auc = 0.657277
Epoch 285
Validation auc = 0.665728
Epoch 286
Validation auc = 0.668075
Epoch 287
Validation auc = 0.668075
Epoch 288
Validation auc = 0.670892
Epoch 289
Loss = 1.3440e-01, PNorm = 25.0877, GNorm = 2.8132, lr_0 = 6.1671e-04
Validation auc = 0.669484
Epoch 290
Validation auc = 0.666667
Epoch 291
Validation auc = 0.668545
Epoch 292
Validation auc = 0.669484
Epoch 293
Validation auc = 0.671831
Epoch 294
Loss = 1.9058e-01, PNorm = 25.1518, GNorm = 1.5249, lr_0 = 6.1144e-04
Validation auc = 0.664319
Epoch 295
Validation auc = 0.664789
Epoch 296
Validation auc = 0.667136
Epoch 297
Validation auc = 0.666197
Epoch 298
Validation auc = 0.665258
Epoch 299
Loss = 1.3161e-01, PNorm = 25.2124, GNorm = 4.2836, lr_0 = 6.0621e-04
Validation auc = 0.668545
Model 0 best validation auc = 0.750704 on epoch 23
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.875376
Ensemble test auc = 0.875376
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.532864
Epoch 1
Validation auc = 0.615493
Epoch 2
Validation auc = 0.625822
Epoch 3
Validation auc = 0.661033
Epoch 4
Loss = 7.2141e-01, PNorm = 20.9844, GNorm = 6.7878, lr_0 = 6.5000e-04
Validation auc = 0.674178
Epoch 5
Validation auc = 0.639906
Epoch 6
Validation auc = 0.619718
Epoch 7
Validation auc = 0.607981
Epoch 8
Validation auc = 0.593897
Epoch 9
Loss = 6.1985e-01, PNorm = 21.0291, GNorm = 10.9234, lr_0 = 9.9743e-04
Validation auc = 0.599061
Epoch 10
Validation auc = 0.633803
Epoch 11
Validation auc = 0.637559
Epoch 12
Validation auc = 0.646009
Epoch 13
Validation auc = 0.671831
Epoch 14
Loss = 4.8748e-01, PNorm = 21.0836, GNorm = 2.9733, lr_0 = 9.8890e-04
Validation auc = 0.681221
Epoch 15
Validation auc = 0.671362
Epoch 16
Validation auc = 0.692019
Epoch 17
Validation auc = 0.695305
Epoch 18
Validation auc = 0.705164
Epoch 19
Loss = 4.7073e-01, PNorm = 21.1486, GNorm = 2.8583, lr_0 = 9.8045e-04
Validation auc = 0.715962
Epoch 20
Validation auc = 0.722066
Epoch 21
Validation auc = 0.725822
Epoch 22
Validation auc = 0.733803
Epoch 23
Validation auc = 0.738028
Epoch 24
Loss = 3.8715e-01, PNorm = 21.2155, GNorm = 2.7796, lr_0 = 9.7207e-04
Validation auc = 0.749296
Epoch 25
Validation auc = 0.749765
Epoch 26
Validation auc = 0.746948
Epoch 27
Validation auc = 0.733803
Epoch 28
Validation auc = 0.734742
Epoch 29
Loss = 3.4540e-01, PNorm = 21.2868, GNorm = 3.6901, lr_0 = 9.6376e-04
Validation auc = 0.729577
Epoch 30
Validation auc = 0.722535
Epoch 31
Validation auc = 0.723474
Epoch 32
Validation auc = 0.722535
Epoch 33
Validation auc = 0.715023
Epoch 34
Loss = 3.7722e-01, PNorm = 21.3626, GNorm = 7.0358, lr_0 = 9.5552e-04
Validation auc = 0.713615
Epoch 35
Validation auc = 0.712207
Epoch 36
Validation auc = 0.711268
Epoch 37
Validation auc = 0.707512
Epoch 38
Validation auc = 0.702347
Epoch 39
Loss = 4.1857e-01, PNorm = 21.4383, GNorm = 4.2791, lr_0 = 9.4735e-04
Validation auc = 0.712207
Epoch 40
Validation auc = 0.710329
Epoch 41
Validation auc = 0.707512
Epoch 42
Validation auc = 0.695775
Epoch 43
Validation auc = 0.708451
Epoch 44
Loss = 4.4466e-01, PNorm = 21.5070, GNorm = 6.2705, lr_0 = 9.3925e-04
Validation auc = 0.709859
Epoch 45
Validation auc = 0.707512
Epoch 46
Validation auc = 0.700000
Epoch 47
Validation auc = 0.699061
Epoch 48
Validation auc = 0.686854
Epoch 49
Loss = 3.0445e-01, PNorm = 21.5772, GNorm = 2.9229, lr_0 = 9.3122e-04
Validation auc = 0.667136
Epoch 50
Validation auc = 0.652113
Epoch 51
Validation auc = 0.658685
Epoch 52
Validation auc = 0.666667
Epoch 53
Validation auc = 0.661502
Epoch 54
Loss = 2.7521e-01, PNorm = 21.6527, GNorm = 4.3682, lr_0 = 9.2326e-04
Validation auc = 0.683568
Epoch 55
Validation auc = 0.691549
Epoch 56
Validation auc = 0.686385
Epoch 57
Validation auc = 0.681221
Epoch 58
Validation auc = 0.680282
Epoch 59
Loss = 3.2547e-01, PNorm = 21.7292, GNorm = 3.2721, lr_0 = 9.1537e-04
Validation auc = 0.685446
Epoch 60
Validation auc = 0.685446
Epoch 61
Validation auc = 0.689671
Epoch 62
Validation auc = 0.682160
Epoch 63
Validation auc = 0.684038
Epoch 64
Loss = 2.8687e-01, PNorm = 21.8169, GNorm = 1.5328, lr_0 = 9.0754e-04
Validation auc = 0.683099
Epoch 65
Validation auc = 0.683568
Epoch 66
Validation auc = 0.684038
Epoch 67
Validation auc = 0.685915
Epoch 68
Validation auc = 0.686385
Epoch 69
Loss = 3.3387e-01, PNorm = 21.9048, GNorm = 3.0054, lr_0 = 8.9978e-04
Validation auc = 0.691080
Epoch 70
Validation auc = 0.694836
Epoch 71
Validation auc = 0.688263
Epoch 72
Validation auc = 0.700939
Epoch 73
Validation auc = 0.704225
Epoch 74
Loss = 2.6139e-01, PNorm = 21.9886, GNorm = 2.0963, lr_0 = 8.9209e-04
Validation auc = 0.704225
Epoch 75
Validation auc = 0.699531
Epoch 76
Validation auc = 0.700469
Epoch 77
Validation auc = 0.704695
Epoch 78
Validation auc = 0.701408
Epoch 79
Loss = 2.4925e-01, PNorm = 22.0790, GNorm = 2.4656, lr_0 = 8.8447e-04
Validation auc = 0.693897
Epoch 80
Validation auc = 0.694836
Epoch 81
Validation auc = 0.698122
Epoch 82
Validation auc = 0.698592
Epoch 83
Validation auc = 0.702817
Epoch 84
Loss = 2.2764e-01, PNorm = 22.1523, GNorm = 1.8829, lr_0 = 8.7691e-04
Validation auc = 0.704225
Epoch 85
Validation auc = 0.705634
Epoch 86
Validation auc = 0.705164
Epoch 87
Validation auc = 0.702817
Epoch 88
Validation auc = 0.700939
Epoch 89
Loss = 2.3115e-01, PNorm = 22.2210, GNorm = 2.2698, lr_0 = 8.6941e-04
Validation auc = 0.705634
Epoch 90
Validation auc = 0.705634
Epoch 91
Validation auc = 0.703756
Epoch 92
Validation auc = 0.705634
Epoch 93
Validation auc = 0.701878
Epoch 94
Loss = 2.3782e-01, PNorm = 22.3004, GNorm = 1.5703, lr_0 = 8.6198e-04
Validation auc = 0.699531
Epoch 95
Validation auc = 0.692019
Epoch 96
Validation auc = 0.694366
Epoch 97
Validation auc = 0.690141
Epoch 98
Validation auc = 0.688732
Epoch 99
Loss = 2.6190e-01, PNorm = 22.3760, GNorm = 2.9840, lr_0 = 8.5461e-04
Validation auc = 0.683099
Epoch 100
Validation auc = 0.682160
Epoch 101
Validation auc = 0.674648
Epoch 102
Validation auc = 0.666667
Epoch 103
Validation auc = 0.668075
Epoch 104
Loss = 2.4820e-01, PNorm = 22.4520, GNorm = 5.7320, lr_0 = 8.4730e-04
Validation auc = 0.672770
Epoch 105
Validation auc = 0.672300
Epoch 106
Validation auc = 0.671362
Epoch 107
Validation auc = 0.675587
Epoch 108
Validation auc = 0.676995
Epoch 109
Loss = 2.4471e-01, PNorm = 22.5350, GNorm = 1.7853, lr_0 = 8.4006e-04
Validation auc = 0.673239
Epoch 110
Validation auc = 0.673709
Epoch 111
Validation auc = 0.672770
Epoch 112
Validation auc = 0.679343
Epoch 113
Validation auc = 0.685446
Epoch 114
Loss = 2.4368e-01, PNorm = 22.6128, GNorm = 1.8349, lr_0 = 8.3288e-04
Validation auc = 0.684507
Epoch 115
Validation auc = 0.686854
Epoch 116
Validation auc = 0.690141
Epoch 117
Validation auc = 0.687324
Epoch 118
Validation auc = 0.679812
Epoch 119
Loss = 2.1616e-01, PNorm = 22.6876, GNorm = 2.9721, lr_0 = 8.2576e-04
Validation auc = 0.666667
Epoch 120
Validation auc = 0.657277
Epoch 121
Validation auc = 0.659624
Epoch 122
Validation auc = 0.661502
Epoch 123
Validation auc = 0.670423
Epoch 124
Loss = 2.3334e-01, PNorm = 22.7701, GNorm = 1.8024, lr_0 = 8.1870e-04
Validation auc = 0.677465
Epoch 125
Validation auc = 0.686385
Epoch 126
Validation auc = 0.682160
Epoch 127
Validation auc = 0.673239
Epoch 128
Validation auc = 0.663850
Epoch 129
Loss = 3.1183e-01, PNorm = 22.8554, GNorm = 8.3791, lr_0 = 8.1170e-04
Validation auc = 0.632394
Epoch 130
Validation auc = 0.611737
Epoch 131
Validation auc = 0.623944
Epoch 132
Validation auc = 0.650704
Epoch 133
Validation auc = 0.656338
Epoch 134
Loss = 1.9955e-01, PNorm = 22.9460, GNorm = 1.5702, lr_0 = 8.0476e-04
Validation auc = 0.662441
Epoch 135
Validation auc = 0.664319
Epoch 136
Validation auc = 0.669953
Epoch 137
Validation auc = 0.677934
Epoch 138
Validation auc = 0.674648
Epoch 139
Loss = 2.7823e-01, PNorm = 23.0319, GNorm = 2.1642, lr_0 = 7.9788e-04
Validation auc = 0.667606
Epoch 140
Validation auc = 0.649296
Epoch 141
Validation auc = 0.648826
Epoch 142
Validation auc = 0.655869
Epoch 143
Validation auc = 0.663850
Epoch 144
Loss = 2.4181e-01, PNorm = 23.1195, GNorm = 1.5692, lr_0 = 7.9106e-04
Validation auc = 0.669953
Epoch 145
Validation auc = 0.684507
Epoch 146
Validation auc = 0.693427
Epoch 147
Validation auc = 0.695305
Epoch 148
Validation auc = 0.699061
Epoch 149
Loss = 2.3289e-01, PNorm = 23.2171, GNorm = 0.9401, lr_0 = 7.8430e-04
Validation auc = 0.695775
Epoch 150
Validation auc = 0.692958
Epoch 151
Validation auc = 0.684038
Epoch 152
Validation auc = 0.672300
Epoch 153
Validation auc = 0.658685
Epoch 154
Loss = 3.6301e-01, PNorm = 23.2991, GNorm = 5.4504, lr_0 = 7.7759e-04
Validation auc = 0.680282
Epoch 155
Validation auc = 0.690610
Epoch 156
Validation auc = 0.690610
Epoch 157
Validation auc = 0.689671
Epoch 158
Validation auc = 0.683099
Epoch 159
Loss = 1.9830e-01, PNorm = 23.3756, GNorm = 2.7170, lr_0 = 7.7095e-04
Validation auc = 0.679812
Epoch 160
Validation auc = 0.668545
Epoch 161
Validation auc = 0.654930
Epoch 162
Validation auc = 0.644131
Epoch 163
Validation auc = 0.635211
Epoch 164
Loss = 2.0534e-01, PNorm = 23.4536, GNorm = 1.2818, lr_0 = 7.6436e-04
Validation auc = 0.620657
Epoch 165
Validation auc = 0.618779
Epoch 166
Validation auc = 0.643662
Epoch 167
Validation auc = 0.660094
Epoch 168
Validation auc = 0.664319
Epoch 169
Loss = 1.9287e-01, PNorm = 23.5317, GNorm = 1.1203, lr_0 = 7.5782e-04
Validation auc = 0.664319
Epoch 170
Validation auc = 0.665728
Epoch 171
Validation auc = 0.661033
Epoch 172
Validation auc = 0.662441
Epoch 173
Validation auc = 0.658685
Epoch 174
Loss = 3.4002e-01, PNorm = 23.6034, GNorm = 4.3296, lr_0 = 7.5134e-04
Validation auc = 0.657746
Epoch 175
Validation auc = 0.661502
Epoch 176
Validation auc = 0.661972
Epoch 177
Validation auc = 0.661972
Epoch 178
Validation auc = 0.661033
Epoch 179
Loss = 1.9302e-01, PNorm = 23.6695, GNorm = 2.4526, lr_0 = 7.4492e-04
Validation auc = 0.663850
Epoch 180
Validation auc = 0.667136
Epoch 181
Validation auc = 0.664319
Epoch 182
Validation auc = 0.659155
Epoch 183
Validation auc = 0.654460
Epoch 184
Loss = 1.8568e-01, PNorm = 23.7406, GNorm = 1.7354, lr_0 = 7.3855e-04
Validation auc = 0.660094
Epoch 185
Validation auc = 0.659155
Epoch 186
Validation auc = 0.665728
Epoch 187
Validation auc = 0.664319
Epoch 188
Validation auc = 0.659624
Epoch 189
Loss = 1.6466e-01, PNorm = 23.8217, GNorm = 1.9501, lr_0 = 7.3224e-04
Validation auc = 0.660563
Epoch 190
Validation auc = 0.667606
Epoch 191
Validation auc = 0.669484
Epoch 192
Validation auc = 0.660094
Epoch 193
Validation auc = 0.653521
Epoch 194
Loss = 1.4627e-01, PNorm = 23.8948, GNorm = 2.0571, lr_0 = 7.2598e-04
Validation auc = 0.656338
Epoch 195
Validation auc = 0.664789
Epoch 196
Validation auc = 0.664319
Epoch 197
Validation auc = 0.667606
Epoch 198
Validation auc = 0.657746
Epoch 199
Loss = 1.3989e-01, PNorm = 23.9574, GNorm = 0.9599, lr_0 = 7.1977e-04
Validation auc = 0.656808
Epoch 200
Validation auc = 0.662441
Epoch 201
Validation auc = 0.671831
Epoch 202
Validation auc = 0.676056
Epoch 203
Validation auc = 0.681690
Epoch 204
Loss = 2.0176e-01, PNorm = 24.0207, GNorm = 1.7381, lr_0 = 7.1362e-04
Validation auc = 0.682160
Epoch 205
Validation auc = 0.676526
Epoch 206
Validation auc = 0.676056
Epoch 207
Validation auc = 0.678873
Epoch 208
Validation auc = 0.670892
Epoch 209
Loss = 1.4296e-01, PNorm = 24.0865, GNorm = 1.3278, lr_0 = 7.0752e-04
Validation auc = 0.668075
Epoch 210
Validation auc = 0.657746
Epoch 211
Validation auc = 0.653052
Epoch 212
Validation auc = 0.654460
Epoch 213
Validation auc = 0.659155
Epoch 214
Loss = 1.8011e-01, PNorm = 24.1381, GNorm = 2.1541, lr_0 = 7.0147e-04
Validation auc = 0.654460
Epoch 215
Validation auc = 0.654460
Epoch 216
Validation auc = 0.646479
Epoch 217
Validation auc = 0.642254
Epoch 218
Validation auc = 0.645540
Epoch 219
Loss = 1.4717e-01, PNorm = 24.1897, GNorm = 1.5397, lr_0 = 6.9548e-04
Validation auc = 0.650235
Epoch 220
Validation auc = 0.653052
Epoch 221
Validation auc = 0.661502
Epoch 222
Validation auc = 0.664789
Epoch 223
Validation auc = 0.665728
Epoch 224
Loss = 1.2972e-01, PNorm = 24.2657, GNorm = 1.3562, lr_0 = 6.8953e-04
Validation auc = 0.667606
Epoch 225
Validation auc = 0.667136
Epoch 226
Validation auc = 0.668545
Epoch 227
Validation auc = 0.669484
Epoch 228
Validation auc = 0.663850
Epoch 229
Loss = 1.0455e-01, PNorm = 24.3377, GNorm = 1.4052, lr_0 = 6.8364e-04
Validation auc = 0.649296
Epoch 230
Validation auc = 0.646479
Epoch 231
Validation auc = 0.646479
Epoch 232
Validation auc = 0.643662
Epoch 233
Validation auc = 0.641315
Epoch 234
Loss = 1.6852e-01, PNorm = 24.4115, GNorm = 1.5624, lr_0 = 6.7779e-04
Validation auc = 0.639437
Epoch 235
Validation auc = 0.636620
Epoch 236
Validation auc = 0.656338
Epoch 237
Validation auc = 0.671362
Epoch 238
Validation auc = 0.671831
Epoch 239
Loss = 1.8781e-01, PNorm = 24.4733, GNorm = 1.6216, lr_0 = 6.7200e-04
Validation auc = 0.672300
Epoch 240
Validation auc = 0.674648
Epoch 241
Validation auc = 0.673239
Epoch 242
Validation auc = 0.674648
Epoch 243
Validation auc = 0.670423
Epoch 244
Loss = 1.2211e-01, PNorm = 24.5363, GNorm = 2.4707, lr_0 = 6.6625e-04
Validation auc = 0.662911
Epoch 245
Validation auc = 0.649765
Epoch 246
Validation auc = 0.651174
Epoch 247
Validation auc = 0.663380
Epoch 248
Validation auc = 0.661972
Epoch 249
Loss = 1.1512e-01, PNorm = 24.5992, GNorm = 1.1878, lr_0 = 6.6056e-04
Validation auc = 0.659624
Epoch 250
Validation auc = 0.652113
Epoch 251
Validation auc = 0.660094
Epoch 252
Validation auc = 0.661502
Epoch 253
Validation auc = 0.667606
Epoch 254
Loss = 1.6172e-01, PNorm = 24.6535, GNorm = 3.6157, lr_0 = 6.5491e-04
Validation auc = 0.664789
Epoch 255
Validation auc = 0.661972
Epoch 256
Validation auc = 0.663850
Epoch 257
Validation auc = 0.669484
Epoch 258
Validation auc = 0.677934
Epoch 259
Loss = 1.6685e-01, PNorm = 24.7096, GNorm = 3.1626, lr_0 = 6.4931e-04
Validation auc = 0.676995
Epoch 260
Validation auc = 0.672770
Epoch 261
Validation auc = 0.662441
Epoch 262
Validation auc = 0.657746
Epoch 263
Validation auc = 0.649296
Epoch 264
Loss = 1.4794e-01, PNorm = 24.7637, GNorm = 2.7332, lr_0 = 6.4376e-04
Validation auc = 0.650704
Epoch 265
Validation auc = 0.657746
Epoch 266
Validation auc = 0.651643
Epoch 267
Validation auc = 0.658216
Epoch 268
Validation auc = 0.665728
Epoch 269
Loss = 1.5536e-01, PNorm = 24.8240, GNorm = 4.7655, lr_0 = 6.3826e-04
Validation auc = 0.661502
Epoch 270
Validation auc = 0.651174
Epoch 271
Validation auc = 0.646009
Epoch 272
Validation auc = 0.648826
Epoch 273
Validation auc = 0.652113
Epoch 274
Loss = 1.4373e-01, PNorm = 24.8817, GNorm = 1.5551, lr_0 = 6.3280e-04
Validation auc = 0.642723
Epoch 275
Validation auc = 0.635681
Epoch 276
Validation auc = 0.627700
Epoch 277
Validation auc = 0.630986
Epoch 278
Validation auc = 0.633333
Epoch 279
Loss = 1.5412e-01, PNorm = 24.9362, GNorm = 1.4027, lr_0 = 6.2739e-04
Validation auc = 0.632394
Epoch 280
Validation auc = 0.641315
Epoch 281
Validation auc = 0.656808
Epoch 282
Validation auc = 0.661972
Epoch 283
Validation auc = 0.664319
Epoch 284
Loss = 1.1205e-01, PNorm = 24.9905, GNorm = 1.3518, lr_0 = 6.2203e-04
Validation auc = 0.664319
Epoch 285
Validation auc = 0.668075
Epoch 286
Validation auc = 0.665258
Epoch 287
Validation auc = 0.664319
Epoch 288
Validation auc = 0.663380
Epoch 289
Loss = 1.1046e-01, PNorm = 25.0440, GNorm = 1.3096, lr_0 = 6.1671e-04
Validation auc = 0.668075
Epoch 290
Validation auc = 0.666197
Epoch 291
Validation auc = 0.665258
Epoch 292
Validation auc = 0.660563
Epoch 293
Validation auc = 0.648826
Epoch 294
Loss = 1.3975e-01, PNorm = 25.0942, GNorm = 2.4159, lr_0 = 6.1144e-04
Validation auc = 0.645070
Epoch 295
Validation auc = 0.654930
Epoch 296
Validation auc = 0.659624
Epoch 297
Validation auc = 0.661033
Epoch 298
Validation auc = 0.660094
Epoch 299
Loss = 1.7858e-01, PNorm = 25.1436, GNorm = 2.0755, lr_0 = 6.0621e-04
Validation auc = 0.662441
Model 0 best validation auc = 0.749765 on epoch 25
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.873570
Ensemble test auc = 0.873570
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.512676
Epoch 1
Validation auc = 0.583099
Epoch 2
Validation auc = 0.606573
Epoch 3
Validation auc = 0.660563
Epoch 4
Loss = 7.5012e-01, PNorm = 20.9836, GNorm = 2.3410, lr_0 = 6.5000e-04
Validation auc = 0.671831
Epoch 5
Validation auc = 0.622535
Epoch 6
Validation auc = 0.616432
Epoch 7
Validation auc = 0.624883
Epoch 8
Validation auc = 0.625822
Epoch 9
Loss = 5.9588e-01, PNorm = 21.0264, GNorm = 6.9154, lr_0 = 9.9743e-04
Validation auc = 0.653521
Epoch 10
Validation auc = 0.684038
Epoch 11
Validation auc = 0.688263
Epoch 12
Validation auc = 0.677465
Epoch 13
Validation auc = 0.678873
Epoch 14
Loss = 4.0141e-01, PNorm = 21.0860, GNorm = 2.6460, lr_0 = 9.8890e-04
Validation auc = 0.686854
Epoch 15
Validation auc = 0.692488
Epoch 16
Validation auc = 0.695305
Epoch 17
Validation auc = 0.687793
Epoch 18
Validation auc = 0.686854
Epoch 19
Loss = 4.5614e-01, PNorm = 21.1481, GNorm = 5.9175, lr_0 = 9.8045e-04
Validation auc = 0.689671
Epoch 20
Validation auc = 0.688263
Epoch 21
Validation auc = 0.695775
Epoch 22
Validation auc = 0.712207
Epoch 23
Validation auc = 0.713615
Epoch 24
Loss = 3.4090e-01, PNorm = 21.2113, GNorm = 1.8613, lr_0 = 9.7207e-04
Validation auc = 0.702347
Epoch 25
Validation auc = 0.711737
Epoch 26
Validation auc = 0.709390
Epoch 27
Validation auc = 0.701408
Epoch 28
Validation auc = 0.692488
Epoch 29
Loss = 3.0410e-01, PNorm = 21.2788, GNorm = 2.6639, lr_0 = 9.6376e-04
Validation auc = 0.684977
Epoch 30
Validation auc = 0.682629
Epoch 31
Validation auc = 0.672300
Epoch 32
Validation auc = 0.671362
Epoch 33
Validation auc = 0.670423
Epoch 34
Loss = 3.3938e-01, PNorm = 21.3520, GNorm = 6.6639, lr_0 = 9.5552e-04
Validation auc = 0.670892
Epoch 35
Validation auc = 0.665728
Epoch 36
Validation auc = 0.670892
Epoch 37
Validation auc = 0.676526
Epoch 38
Validation auc = 0.672300
Epoch 39
Loss = 4.1602e-01, PNorm = 21.4248, GNorm = 9.4023, lr_0 = 9.4735e-04
Validation auc = 0.673709
Epoch 40
Validation auc = 0.678873
Epoch 41
Validation auc = 0.683568
Epoch 42
Validation auc = 0.670423
Epoch 43
Validation auc = 0.687324
Epoch 44
Loss = 3.5917e-01, PNorm = 21.5034, GNorm = 5.1707, lr_0 = 9.3925e-04
Validation auc = 0.692488
Epoch 45
Validation auc = 0.692958
Epoch 46
Validation auc = 0.686385
Epoch 47
Validation auc = 0.695305
Epoch 48
Validation auc = 0.693897
Epoch 49
Loss = 3.2325e-01, PNorm = 21.5909, GNorm = 1.6965, lr_0 = 9.3122e-04
Validation auc = 0.693897
Epoch 50
Validation auc = 0.656808
Epoch 51
Validation auc = 0.659155
Epoch 52
Validation auc = 0.653521
Epoch 53
Validation auc = 0.636620
Epoch 54
Loss = 3.1872e-01, PNorm = 21.6690, GNorm = 2.9326, lr_0 = 9.2326e-04
Validation auc = 0.633803
Epoch 55
Validation auc = 0.646009
Epoch 56
Validation auc = 0.650704
Epoch 57
Validation auc = 0.660094
Epoch 58
Validation auc = 0.653991
Epoch 59
Loss = 4.0908e-01, PNorm = 21.7469, GNorm = 7.1465, lr_0 = 9.1537e-04
Validation auc = 0.655399
Epoch 60
Validation auc = 0.669014
Epoch 61
Validation auc = 0.677934
Epoch 62
Validation auc = 0.673239
Epoch 63
Validation auc = 0.680282
Epoch 64
Loss = 2.5201e-01, PNorm = 21.8222, GNorm = 2.8507, lr_0 = 9.0754e-04
Validation auc = 0.683099
Epoch 65
Validation auc = 0.688263
Epoch 66
Validation auc = 0.684977
Epoch 67
Validation auc = 0.677934
Epoch 68
Validation auc = 0.681690
Epoch 69
Loss = 2.9098e-01, PNorm = 21.9040, GNorm = 4.4272, lr_0 = 8.9978e-04
Validation auc = 0.680751
Epoch 70
Validation auc = 0.675587
Epoch 71
Validation auc = 0.656808
Epoch 72
Validation auc = 0.667606
Epoch 73
Validation auc = 0.674648
Epoch 74
Loss = 2.7164e-01, PNorm = 21.9797, GNorm = 2.3429, lr_0 = 8.9209e-04
Validation auc = 0.674648
Epoch 75
Validation auc = 0.670423
Epoch 76
Validation auc = 0.662441
Epoch 77
Validation auc = 0.677465
Epoch 78
Validation auc = 0.680751
Epoch 79
Loss = 2.7568e-01, PNorm = 22.0624, GNorm = 1.9265, lr_0 = 8.8447e-04
Validation auc = 0.678404
Epoch 80
Validation auc = 0.679343
Epoch 81
Validation auc = 0.677465
Epoch 82
Validation auc = 0.673239
Epoch 83
Validation auc = 0.665728
Epoch 84
Loss = 3.8228e-01, PNorm = 22.1474, GNorm = 7.1318, lr_0 = 8.7691e-04
Validation auc = 0.668075
Epoch 85
Validation auc = 0.670423
Epoch 86
Validation auc = 0.661502
Epoch 87
Validation auc = 0.658216
Epoch 88
Validation auc = 0.657277
Epoch 89
Loss = 2.6977e-01, PNorm = 22.2256, GNorm = 1.5539, lr_0 = 8.6941e-04
Validation auc = 0.666197
Epoch 90
Validation auc = 0.665728
Epoch 91
Validation auc = 0.663850
Epoch 92
Validation auc = 0.640376
Epoch 93
Validation auc = 0.653521
Epoch 94
Loss = 2.8249e-01, PNorm = 22.3130, GNorm = 2.2879, lr_0 = 8.6198e-04
Validation auc = 0.661972
Epoch 95
Validation auc = 0.664789
Epoch 96
Validation auc = 0.662911
Epoch 97
Validation auc = 0.664789
Epoch 98
Validation auc = 0.658216
Epoch 99
Loss = 2.6526e-01, PNorm = 22.3901, GNorm = 1.6005, lr_0 = 8.5461e-04
Validation auc = 0.639906
Epoch 100
Validation auc = 0.630986
Epoch 101
Validation auc = 0.623944
Epoch 102
Validation auc = 0.619718
Epoch 103
Validation auc = 0.654460
Epoch 104
Loss = 2.5744e-01, PNorm = 22.4683, GNorm = 6.0490, lr_0 = 8.4730e-04
Validation auc = 0.672300
Epoch 105
Validation auc = 0.674178
Epoch 106
Validation auc = 0.676056
Epoch 107
Validation auc = 0.678404
Epoch 108
Validation auc = 0.680282
Epoch 109
Loss = 1.9964e-01, PNorm = 22.5515, GNorm = 3.1384, lr_0 = 8.4006e-04
Validation auc = 0.676526
Epoch 110
Validation auc = 0.660094
Epoch 111
Validation auc = 0.658685
Epoch 112
Validation auc = 0.663380
Epoch 113
Validation auc = 0.677934
Epoch 114
Loss = 2.7940e-01, PNorm = 22.6329, GNorm = 1.7529, lr_0 = 8.3288e-04
Validation auc = 0.673239
Epoch 115
Validation auc = 0.682160
Epoch 116
Validation auc = 0.692019
Epoch 117
Validation auc = 0.695775
Epoch 118
Validation auc = 0.696244
Epoch 119
Loss = 2.4327e-01, PNorm = 22.7099, GNorm = 2.1769, lr_0 = 8.2576e-04
Validation auc = 0.690610
Epoch 120
Validation auc = 0.685446
Epoch 121
Validation auc = 0.678404
Epoch 122
Validation auc = 0.664789
Epoch 123
Validation auc = 0.642254
Epoch 124
Loss = 2.0945e-01, PNorm = 22.7816, GNorm = 3.5953, lr_0 = 8.1870e-04
Validation auc = 0.627700
Epoch 125
Validation auc = 0.634742
Epoch 126
Validation auc = 0.643662
Epoch 127
Validation auc = 0.622535
Epoch 128
Validation auc = 0.622535
Epoch 129
Loss = 1.6813e-01, PNorm = 22.8517, GNorm = 0.6981, lr_0 = 8.1170e-04
Validation auc = 0.640376
Epoch 130
Validation auc = 0.658685
Epoch 131
Validation auc = 0.670423
Epoch 132
Validation auc = 0.671362
Epoch 133
Validation auc = 0.674648
Epoch 134
Loss = 2.5501e-01, PNorm = 22.9219, GNorm = 3.4084, lr_0 = 8.0476e-04
Validation auc = 0.667136
Epoch 135
Validation auc = 0.678404
Epoch 136
Validation auc = 0.672770
Epoch 137
Validation auc = 0.655869
Epoch 138
Validation auc = 0.642723
Epoch 139
Loss = 2.9484e-01, PNorm = 23.0088, GNorm = 3.2656, lr_0 = 7.9788e-04
Validation auc = 0.650235
Epoch 140
Validation auc = 0.660563
Epoch 141
Validation auc = 0.653991
Epoch 142
Validation auc = 0.628169
Epoch 143
Validation auc = 0.622066
Epoch 144
Loss = 1.9478e-01, PNorm = 23.0897, GNorm = 0.9870, lr_0 = 7.9106e-04
Validation auc = 0.647418
Epoch 145
Validation auc = 0.664319
Epoch 146
Validation auc = 0.669484
Epoch 147
Validation auc = 0.668075
Epoch 148
Validation auc = 0.665258
Epoch 149
Loss = 1.5704e-01, PNorm = 23.1672, GNorm = 1.7348, lr_0 = 7.8430e-04
Validation auc = 0.665258
Epoch 150
Validation auc = 0.664319
Epoch 151
Validation auc = 0.660094
Epoch 152
Validation auc = 0.664319
Epoch 153
Validation auc = 0.666667
Epoch 154
Loss = 1.8249e-01, PNorm = 23.2379, GNorm = 1.5642, lr_0 = 7.7759e-04
Validation auc = 0.670892
Epoch 155
Validation auc = 0.678404
Epoch 156
Validation auc = 0.674178
Epoch 157
Validation auc = 0.670423
Epoch 158
Validation auc = 0.671831
Epoch 159
Loss = 2.3853e-01, PNorm = 23.3014, GNorm = 2.0871, lr_0 = 7.7095e-04
Validation auc = 0.674178
Epoch 160
Validation auc = 0.677934
Epoch 161
Validation auc = 0.678404
Epoch 162
Validation auc = 0.673239
Epoch 163
Validation auc = 0.666667
Epoch 164
Loss = 2.8632e-01, PNorm = 23.3672, GNorm = 1.8063, lr_0 = 7.6436e-04
Validation auc = 0.668545
Epoch 165
Validation auc = 0.674648
Epoch 166
Validation auc = 0.677934
Epoch 167
Validation auc = 0.678404
Epoch 168
Validation auc = 0.674178
Epoch 169
Loss = 2.2414e-01, PNorm = 23.4364, GNorm = 1.0833, lr_0 = 7.5782e-04
Validation auc = 0.676056
Epoch 170
Validation auc = 0.673239
Epoch 171
Validation auc = 0.673239
Epoch 172
Validation auc = 0.674648
Epoch 173
Validation auc = 0.671831
Epoch 174
Loss = 1.6294e-01, PNorm = 23.5067, GNorm = 1.2199, lr_0 = 7.5134e-04
Validation auc = 0.674178
Epoch 175
Validation auc = 0.673239
Epoch 176
Validation auc = 0.675117
Epoch 177
Validation auc = 0.671831
Epoch 178
Validation auc = 0.674648
Epoch 179
Loss = 1.6413e-01, PNorm = 23.5691, GNorm = 3.4810, lr_0 = 7.4492e-04
Validation auc = 0.676056
Epoch 180
Validation auc = 0.675587
Epoch 181
Validation auc = 0.668545
Epoch 182
Validation auc = 0.670423
Epoch 183
Validation auc = 0.669484
Epoch 184
Loss = 1.6667e-01, PNorm = 23.6244, GNorm = 2.3896, lr_0 = 7.3855e-04
Validation auc = 0.671362
Epoch 185
Validation auc = 0.667136
Epoch 186
Validation auc = 0.669014
Epoch 187
Validation auc = 0.667606
Epoch 188
Validation auc = 0.664789
Epoch 189
Loss = 1.3747e-01, PNorm = 23.6795, GNorm = 1.9968, lr_0 = 7.3224e-04
Validation auc = 0.653521
Epoch 190
Validation auc = 0.648357
Epoch 191
Validation auc = 0.646479
Epoch 192
Validation auc = 0.653991
Epoch 193
Validation auc = 0.651643
Epoch 194
Loss = 1.4612e-01, PNorm = 23.7377, GNorm = 1.1112, lr_0 = 7.2598e-04
Validation auc = 0.649296
Epoch 195
Validation auc = 0.654930
Epoch 196
Validation auc = 0.664789
Epoch 197
Validation auc = 0.664789
Epoch 198
Validation auc = 0.664789
Epoch 199
Loss = 2.6577e-01, PNorm = 23.8189, GNorm = 4.9017, lr_0 = 7.1977e-04
Validation auc = 0.658685
Epoch 200
Validation auc = 0.656808
Epoch 201
Validation auc = 0.655869
Epoch 202
Validation auc = 0.654930
Epoch 203
Validation auc = 0.654930
Epoch 204
Loss = 1.5932e-01, PNorm = 23.8908, GNorm = 3.2833, lr_0 = 7.1362e-04
Validation auc = 0.653991
Epoch 205
Validation auc = 0.649765
Epoch 206
Validation auc = 0.647418
Epoch 207
Validation auc = 0.648357
Epoch 208
Validation auc = 0.647418
Epoch 209
Loss = 1.7787e-01, PNorm = 23.9513, GNorm = 2.1292, lr_0 = 7.0752e-04
Validation auc = 0.643662
Epoch 210
Validation auc = 0.641315
Epoch 211
Validation auc = 0.651174
Epoch 212
Validation auc = 0.657277
Epoch 213
Validation auc = 0.656808
Epoch 214
Loss = 3.3199e-01, PNorm = 24.0123, GNorm = 2.4189, lr_0 = 7.0147e-04
Validation auc = 0.650704
Epoch 215
Validation auc = 0.647887
Epoch 216
Validation auc = 0.639437
Epoch 217
Validation auc = 0.646948
Epoch 218
Validation auc = 0.652113
Epoch 219
Loss = 1.4463e-01, PNorm = 24.0775, GNorm = 1.6170, lr_0 = 6.9548e-04
Validation auc = 0.652113
Epoch 220
Validation auc = 0.659155
Epoch 221
Validation auc = 0.668075
Epoch 222
Validation auc = 0.666197
Epoch 223
Validation auc = 0.669484
Epoch 224
Loss = 1.6000e-01, PNorm = 24.1453, GNorm = 3.5689, lr_0 = 6.8953e-04
Validation auc = 0.667606
Epoch 225
Validation auc = 0.665258
Epoch 226
Validation auc = 0.664789
Epoch 227
Validation auc = 0.657746
Epoch 228
Validation auc = 0.652582
Epoch 229
Loss = 1.7341e-01, PNorm = 24.2053, GNorm = 1.9976, lr_0 = 6.8364e-04
Validation auc = 0.639437
Epoch 230
Validation auc = 0.646009
Epoch 231
Validation auc = 0.653521
Epoch 232
Validation auc = 0.658216
Epoch 233
Validation auc = 0.651643
Epoch 234
Loss = 1.8412e-01, PNorm = 24.2650, GNorm = 5.4188, lr_0 = 6.7779e-04
Validation auc = 0.655399
Epoch 235
Validation auc = 0.660563
Epoch 236
Validation auc = 0.659155
Epoch 237
Validation auc = 0.654460
Epoch 238
Validation auc = 0.648357
Epoch 239
Loss = 1.9526e-01, PNorm = 24.3221, GNorm = 2.9866, lr_0 = 6.7200e-04
Validation auc = 0.643192
Epoch 240
Validation auc = 0.653991
Epoch 241
Validation auc = 0.662911
Epoch 242
Validation auc = 0.664319
Epoch 243
Validation auc = 0.661972
Epoch 244
Loss = 2.0589e-01, PNorm = 24.3688, GNorm = 3.9585, lr_0 = 6.6625e-04
Validation auc = 0.665258
Epoch 245
Validation auc = 0.668545
Epoch 246
Validation auc = 0.666197
Epoch 247
Validation auc = 0.644601
Epoch 248
Validation auc = 0.631455
Epoch 249
Loss = 1.2776e-01, PNorm = 24.4352, GNorm = 0.8850, lr_0 = 6.6056e-04
Validation auc = 0.627230
Epoch 250
Validation auc = 0.633803
Epoch 251
Validation auc = 0.638967
Epoch 252
Validation auc = 0.640845
Epoch 253
Validation auc = 0.635211
Epoch 254
Loss = 1.5471e-01, PNorm = 24.4982, GNorm = 1.7139, lr_0 = 6.5491e-04
Validation auc = 0.638967
Epoch 255
Validation auc = 0.635211
Epoch 256
Validation auc = 0.623944
Epoch 257
Validation auc = 0.617840
Epoch 258
Validation auc = 0.612676
Epoch 259
Loss = 1.6502e-01, PNorm = 24.5587, GNorm = 2.4148, lr_0 = 6.4931e-04
Validation auc = 0.614085
Epoch 260
Validation auc = 0.626291
Epoch 261
Validation auc = 0.639437
Epoch 262
Validation auc = 0.645540
Epoch 263
Validation auc = 0.645070
Epoch 264
Loss = 2.6428e-01, PNorm = 24.6187, GNorm = 6.5989, lr_0 = 6.4376e-04
Validation auc = 0.668545
Epoch 265
Validation auc = 0.680282
Epoch 266
Validation auc = 0.677934
Epoch 267
Validation auc = 0.681221
Epoch 268
Validation auc = 0.684977
Epoch 269
Loss = 1.7241e-01, PNorm = 24.6792, GNorm = 1.7336, lr_0 = 6.3826e-04
Validation auc = 0.690610
Epoch 270
Validation auc = 0.691080
Epoch 271
Validation auc = 0.693897
Epoch 272
Validation auc = 0.690610
Epoch 273
Validation auc = 0.687324
Epoch 274
Loss = 2.6639e-01, PNorm = 24.7412, GNorm = 5.3030, lr_0 = 6.3280e-04
Validation auc = 0.676526
Epoch 275
Validation auc = 0.673239
Epoch 276
Validation auc = 0.677465
Epoch 277
Validation auc = 0.674648
Epoch 278
Validation auc = 0.672300
Epoch 279
Loss = 2.0510e-01, PNorm = 24.8107, GNorm = 1.6144, lr_0 = 6.2739e-04
Validation auc = 0.672300
Epoch 280
Validation auc = 0.676526
Epoch 281
Validation auc = 0.675587
Epoch 282
Validation auc = 0.679812
Epoch 283
Validation auc = 0.678404
Epoch 284
Loss = 1.7129e-01, PNorm = 24.8752, GNorm = 1.9502, lr_0 = 6.2203e-04
Validation auc = 0.680751
Epoch 285
Validation auc = 0.679343
Epoch 286
Validation auc = 0.678404
Epoch 287
Validation auc = 0.681221
Epoch 288
Validation auc = 0.678873
Epoch 289
Loss = 1.6726e-01, PNorm = 24.9311, GNorm = 0.9986, lr_0 = 6.1671e-04
Validation auc = 0.682160
Epoch 290
Validation auc = 0.683099
Epoch 291
Validation auc = 0.684977
Epoch 292
Validation auc = 0.686385
Epoch 293
Validation auc = 0.686854
Epoch 294
Loss = 1.4298e-01, PNorm = 24.9791, GNorm = 0.8907, lr_0 = 6.1144e-04
Validation auc = 0.687793
Epoch 295
Validation auc = 0.689202
Epoch 296
Validation auc = 0.690141
Epoch 297
Validation auc = 0.694366
Epoch 298
Validation auc = 0.694366
Epoch 299
Loss = 1.6940e-01, PNorm = 25.0340, GNorm = 0.9378, lr_0 = 6.0621e-04
Validation auc = 0.693427
Model 0 best validation auc = 0.713615 on epoch 23
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.877784
Ensemble test auc = 0.877784
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.517840
Epoch 1
Validation auc = 0.605634
Epoch 2
Validation auc = 0.622535
Epoch 3
Validation auc = 0.672300
Epoch 4
Loss = 7.6937e-01, PNorm = 20.9842, GNorm = 5.0865, lr_0 = 6.5000e-04
Validation auc = 0.672300
Epoch 5
Validation auc = 0.667136
Epoch 6
Validation auc = 0.625352
Epoch 7
Validation auc = 0.616432
Epoch 8
Validation auc = 0.628638
Epoch 9
Loss = 6.1149e-01, PNorm = 21.0293, GNorm = 3.1253, lr_0 = 9.9743e-04
Validation auc = 0.620188
Epoch 10
Validation auc = 0.615493
Epoch 11
Validation auc = 0.675117
Epoch 12
Validation auc = 0.681221
Epoch 13
Validation auc = 0.663850
Epoch 14
Loss = 4.7696e-01, PNorm = 21.0950, GNorm = 1.8587, lr_0 = 9.8890e-04
Validation auc = 0.670423
Epoch 15
Validation auc = 0.694836
Epoch 16
Validation auc = 0.708920
Epoch 17
Validation auc = 0.724413
Epoch 18
Validation auc = 0.732864
Epoch 19
Loss = 4.1777e-01, PNorm = 21.1613, GNorm = 3.2150, lr_0 = 9.8045e-04
Validation auc = 0.723474
Epoch 20
Validation auc = 0.716901
Epoch 21
Validation auc = 0.735211
Epoch 22
Validation auc = 0.731455
Epoch 23
Validation auc = 0.719718
Epoch 24
Loss = 3.9225e-01, PNorm = 21.2269, GNorm = 1.7580, lr_0 = 9.7207e-04
Validation auc = 0.722066
Epoch 25
Validation auc = 0.722066
Epoch 26
Validation auc = 0.711268
Epoch 27
Validation auc = 0.706103
Epoch 28
Validation auc = 0.708920
Epoch 29
Loss = 3.3615e-01, PNorm = 21.2896, GNorm = 2.1801, lr_0 = 9.6376e-04
Validation auc = 0.708920
Epoch 30
Validation auc = 0.718779
Epoch 31
Validation auc = 0.723474
Epoch 32
Validation auc = 0.728638
Epoch 33
Validation auc = 0.737559
Epoch 34
Loss = 3.5490e-01, PNorm = 21.3608, GNorm = 2.5742, lr_0 = 9.5552e-04
Validation auc = 0.743192
Epoch 35
Validation auc = 0.749765
Epoch 36
Validation auc = 0.741315
Epoch 37
Validation auc = 0.736150
Epoch 38
Validation auc = 0.738967
Epoch 39
Loss = 4.3323e-01, PNorm = 21.4322, GNorm = 2.2661, lr_0 = 9.4735e-04
Validation auc = 0.720188
Epoch 40
Validation auc = 0.716901
Epoch 41
Validation auc = 0.712676
Epoch 42
Validation auc = 0.703756
Epoch 43
Validation auc = 0.697653
Epoch 44
Loss = 3.7461e-01, PNorm = 21.5132, GNorm = 1.8423, lr_0 = 9.3925e-04
Validation auc = 0.689202
Epoch 45
Validation auc = 0.680282
Epoch 46
Validation auc = 0.671362
Epoch 47
Validation auc = 0.671362
Epoch 48
Validation auc = 0.674178
Epoch 49
Loss = 3.4753e-01, PNorm = 21.5933, GNorm = 2.2109, lr_0 = 9.3122e-04
Validation auc = 0.667606
Epoch 50
Validation auc = 0.669484
Epoch 51
Validation auc = 0.672300
Epoch 52
Validation auc = 0.675117
Epoch 53
Validation auc = 0.674178
Epoch 54
Loss = 2.8125e-01, PNorm = 21.6751, GNorm = 1.3583, lr_0 = 9.2326e-04
Validation auc = 0.674178
Epoch 55
Validation auc = 0.678404
Epoch 56
Validation auc = 0.680282
Epoch 57
Validation auc = 0.674648
Epoch 58
Validation auc = 0.659624
Epoch 59
Loss = 3.4493e-01, PNorm = 21.7597, GNorm = 2.1154, lr_0 = 9.1537e-04
Validation auc = 0.671831
Epoch 60
Validation auc = 0.680282
Epoch 61
Validation auc = 0.674648
Epoch 62
Validation auc = 0.664319
Epoch 63
Validation auc = 0.676056
Epoch 64
Loss = 2.9498e-01, PNorm = 21.8452, GNorm = 8.0451, lr_0 = 9.0754e-04
Validation auc = 0.675587
Epoch 65
Validation auc = 0.681221
Epoch 66
Validation auc = 0.690610
Epoch 67
Validation auc = 0.685915
Epoch 68
Validation auc = 0.671831
Epoch 69
Loss = 3.3811e-01, PNorm = 21.9333, GNorm = 2.1582, lr_0 = 8.9978e-04
Validation auc = 0.678404
Epoch 70
Validation auc = 0.679343
Epoch 71
Validation auc = 0.671831
Epoch 72
Validation auc = 0.676526
Epoch 73
Validation auc = 0.681690
Epoch 74
Loss = 2.3727e-01, PNorm = 22.0203, GNorm = 1.5968, lr_0 = 8.9209e-04
Validation auc = 0.678873
Epoch 75
Validation auc = 0.676995
Epoch 76
Validation auc = 0.665728
Epoch 77
Validation auc = 0.659624
Epoch 78
Validation auc = 0.665258
Epoch 79
Loss = 2.6784e-01, PNorm = 22.0950, GNorm = 5.0496, lr_0 = 8.8447e-04
Validation auc = 0.658216
Epoch 80
Validation auc = 0.638967
Epoch 81
Validation auc = 0.670423
Epoch 82
Validation auc = 0.668545
Epoch 83
Validation auc = 0.666667
Epoch 84
Loss = 2.9689e-01, PNorm = 22.1758, GNorm = 3.7165, lr_0 = 8.7691e-04
Validation auc = 0.664319
Epoch 85
Validation auc = 0.661033
Epoch 86
Validation auc = 0.659155
Epoch 87
Validation auc = 0.652582
Epoch 88
Validation auc = 0.653991
Epoch 89
Loss = 2.6605e-01, PNorm = 22.2532, GNorm = 4.2070, lr_0 = 8.6941e-04
Validation auc = 0.662911
Epoch 90
Validation auc = 0.653052
Epoch 91
Validation auc = 0.656338
Epoch 92
Validation auc = 0.653521
Epoch 93
Validation auc = 0.649296
Epoch 94
Loss = 2.9940e-01, PNorm = 22.3412, GNorm = 1.6721, lr_0 = 8.6198e-04
Validation auc = 0.645540
Epoch 95
Validation auc = 0.657746
Epoch 96
Validation auc = 0.660563
Epoch 97
Validation auc = 0.653991
Epoch 98
Validation auc = 0.659155
Epoch 99
Loss = 3.1062e-01, PNorm = 22.4276, GNorm = 2.7680, lr_0 = 8.5461e-04
Validation auc = 0.668075
Epoch 100
Validation auc = 0.661502
Epoch 101
Validation auc = 0.650704
Epoch 102
Validation auc = 0.658216
Epoch 103
Validation auc = 0.672300
Epoch 104
Loss = 2.5093e-01, PNorm = 22.5206, GNorm = 5.4061, lr_0 = 8.4730e-04
Validation auc = 0.671831
Epoch 105
Validation auc = 0.676056
Epoch 106
Validation auc = 0.682629
Epoch 107
Validation auc = 0.683568
Epoch 108
Validation auc = 0.680282
Epoch 109
Loss = 2.1813e-01, PNorm = 22.6091, GNorm = 1.1673, lr_0 = 8.4006e-04
Validation auc = 0.658685
Epoch 110
Validation auc = 0.664789
Epoch 111
Validation auc = 0.673239
Epoch 112
Validation auc = 0.674648
Epoch 113
Validation auc = 0.667136
Epoch 114
Loss = 4.7176e-01, PNorm = 22.6812, GNorm = 5.6102, lr_0 = 8.3288e-04
Validation auc = 0.676056
Epoch 115
Validation auc = 0.679343
Epoch 116
Validation auc = 0.677934
Epoch 117
Validation auc = 0.676056
Epoch 118
Validation auc = 0.671831
Epoch 119
Loss = 4.2540e-01, PNorm = 22.7661, GNorm = 6.6441, lr_0 = 8.2576e-04
Validation auc = 0.679812
Epoch 120
Validation auc = 0.678873
Epoch 121
Validation auc = 0.668075
Epoch 122
Validation auc = 0.634742
Epoch 123
Validation auc = 0.643192
Epoch 124
Loss = 2.2696e-01, PNorm = 22.8589, GNorm = 2.6551, lr_0 = 8.1870e-04
Validation auc = 0.664319
Epoch 125
Validation auc = 0.660563
Epoch 126
Validation auc = 0.636620
Epoch 127
Validation auc = 0.633333
Epoch 128
Validation auc = 0.636150
Epoch 129
Loss = 2.3327e-01, PNorm = 22.9461, GNorm = 4.9890, lr_0 = 8.1170e-04
Validation auc = 0.631925
Epoch 130
Validation auc = 0.627700
Epoch 131
Validation auc = 0.637559
Epoch 132
Validation auc = 0.653052
Epoch 133
Validation auc = 0.662441
Epoch 134
Loss = 2.2191e-01, PNorm = 23.0172, GNorm = 6.3998, lr_0 = 8.0476e-04
Validation auc = 0.651174
Epoch 135
Validation auc = 0.632394
Epoch 136
Validation auc = 0.631455
Epoch 137
Validation auc = 0.642723
Epoch 138
Validation auc = 0.650235
Epoch 139
Loss = 1.4892e-01, PNorm = 23.0921, GNorm = 2.7822, lr_0 = 7.9788e-04
Validation auc = 0.641315
Epoch 140
Validation auc = 0.634272
Epoch 141
Validation auc = 0.639437
Epoch 142
Validation auc = 0.652113
Epoch 143
Validation auc = 0.663380
Epoch 144
Loss = 1.4951e-01, PNorm = 23.1659, GNorm = 2.0210, lr_0 = 7.9106e-04
Validation auc = 0.653052
Epoch 145
Validation auc = 0.639437
Epoch 146
Validation auc = 0.630047
Epoch 147
Validation auc = 0.646479
Epoch 148
Validation auc = 0.645540
Epoch 149
Loss = 2.0183e-01, PNorm = 23.2335, GNorm = 1.9354, lr_0 = 7.8430e-04
Validation auc = 0.646009
Epoch 150
Validation auc = 0.658685
Epoch 151
Validation auc = 0.672300
Epoch 152
Validation auc = 0.682160
Epoch 153
Validation auc = 0.682160
Epoch 154
Loss = 1.7016e-01, PNorm = 23.2983, GNorm = 1.4108, lr_0 = 7.7759e-04
Validation auc = 0.676526
Epoch 155
Validation auc = 0.673239
Epoch 156
Validation auc = 0.666197
Epoch 157
Validation auc = 0.659624
Epoch 158
Validation auc = 0.646479
Epoch 159
Loss = 2.3730e-01, PNorm = 23.3676, GNorm = 3.5777, lr_0 = 7.7095e-04
Validation auc = 0.640845
Epoch 160
Validation auc = 0.646948
Epoch 161
Validation auc = 0.662911
Epoch 162
Validation auc = 0.668075
Epoch 163
Validation auc = 0.670423
Epoch 164
Loss = 1.4412e-01, PNorm = 23.4329, GNorm = 1.0845, lr_0 = 7.6436e-04
Validation auc = 0.668075
Epoch 165
Validation auc = 0.669953
Epoch 166
Validation auc = 0.684507
Epoch 167
Validation auc = 0.682160
Epoch 168
Validation auc = 0.687324
Epoch 169
Loss = 2.4672e-01, PNorm = 23.4963, GNorm = 3.1163, lr_0 = 7.5782e-04
Validation auc = 0.681690
Epoch 170
Validation auc = 0.681690
Epoch 171
Validation auc = 0.683099
Epoch 172
Validation auc = 0.681690
Epoch 173
Validation auc = 0.679812
Epoch 174
Loss = 1.3435e-01, PNorm = 23.5594, GNorm = 0.6037, lr_0 = 7.5134e-04
Validation auc = 0.676995
Epoch 175
Validation auc = 0.669014
Epoch 176
Validation auc = 0.669953
Epoch 177
Validation auc = 0.674648
Epoch 178
Validation auc = 0.677934
Epoch 179
Loss = 1.3579e-01, PNorm = 23.6226, GNorm = 1.1418, lr_0 = 7.4492e-04
Validation auc = 0.676056
Epoch 180
Validation auc = 0.679343
Epoch 181
Validation auc = 0.677934
Epoch 182
Validation auc = 0.677465
Epoch 183
Validation auc = 0.678873
Epoch 184
Loss = 1.2921e-01, PNorm = 23.6839, GNorm = 1.7304, lr_0 = 7.3855e-04
Validation auc = 0.680751
Epoch 185
Validation auc = 0.680751
Epoch 186
Validation auc = 0.681690
Epoch 187
Validation auc = 0.684507
Epoch 188
Validation auc = 0.684038
Epoch 189
Loss = 1.7466e-01, PNorm = 23.7534, GNorm = 1.4087, lr_0 = 7.3224e-04
Validation auc = 0.685446
Epoch 190
Validation auc = 0.686385
Epoch 191
Validation auc = 0.690141
Epoch 192
Validation auc = 0.688732
Epoch 193
Validation auc = 0.697653
Epoch 194
Loss = 1.3618e-01, PNorm = 23.8179, GNorm = 2.2626, lr_0 = 7.2598e-04
Validation auc = 0.706103
Epoch 195
Validation auc = 0.710798
Epoch 196
Validation auc = 0.712676
Epoch 197
Validation auc = 0.711268
Epoch 198
Validation auc = 0.710798
Epoch 199
Loss = 2.3287e-01, PNorm = 23.8886, GNorm = 2.9729, lr_0 = 7.1977e-04
Validation auc = 0.713146
Epoch 200
Validation auc = 0.708451
Epoch 201
Validation auc = 0.703756
Epoch 202
Validation auc = 0.690141
Epoch 203
Validation auc = 0.684507
Epoch 204
Loss = 1.8838e-01, PNorm = 23.9563, GNorm = 2.8317, lr_0 = 7.1362e-04
Validation auc = 0.684507
Epoch 205
Validation auc = 0.678873
Epoch 206
Validation auc = 0.673239
Epoch 207
Validation auc = 0.670423
Epoch 208
Validation auc = 0.671831
Epoch 209
Loss = 1.5984e-01, PNorm = 24.0193, GNorm = 1.0425, lr_0 = 7.0752e-04
Validation auc = 0.674648
Epoch 210
Validation auc = 0.674648
Epoch 211
Validation auc = 0.675117
Epoch 212
Validation auc = 0.676056
Epoch 213
Validation auc = 0.677465
Epoch 214
Loss = 1.8033e-01, PNorm = 24.0849, GNorm = 3.5891, lr_0 = 7.0147e-04
Validation auc = 0.677934
Epoch 215
Validation auc = 0.682629
Epoch 216
Validation auc = 0.678873
Epoch 217
Validation auc = 0.677934
Epoch 218
Validation auc = 0.666667
Epoch 219
Loss = 1.5992e-01, PNorm = 24.1457, GNorm = 1.2803, lr_0 = 6.9548e-04
Validation auc = 0.662441
Epoch 220
Validation auc = 0.667606
Epoch 221
Validation auc = 0.668075
Epoch 222
Validation auc = 0.673239
Epoch 223
Validation auc = 0.670423
Epoch 224
Loss = 1.9231e-01, PNorm = 24.2182, GNorm = 3.0006, lr_0 = 6.8953e-04
Validation auc = 0.671831
Epoch 225
Validation auc = 0.676056
Epoch 226
Validation auc = 0.669014
Epoch 227
Validation auc = 0.662911
Epoch 228
Validation auc = 0.665258
Epoch 229
Loss = 1.2521e-01, PNorm = 24.2811, GNorm = 1.5521, lr_0 = 6.8364e-04
Validation auc = 0.666667
Epoch 230
Validation auc = 0.670892
Epoch 231
Validation auc = 0.670892
Epoch 232
Validation auc = 0.670892
Epoch 233
Validation auc = 0.671362
Epoch 234
Loss = 1.4001e-01, PNorm = 24.3364, GNorm = 3.2033, lr_0 = 6.7779e-04
Validation auc = 0.678404
Epoch 235
Validation auc = 0.679812
Epoch 236
Validation auc = 0.678404
Epoch 237
Validation auc = 0.671831
Epoch 238
Validation auc = 0.666667
Epoch 239
Loss = 2.1131e-01, PNorm = 24.4025, GNorm = 1.8271, lr_0 = 6.7200e-04
Validation auc = 0.662911
Epoch 240
Validation auc = 0.661033
Epoch 241
Validation auc = 0.659155
Epoch 242
Validation auc = 0.661972
Epoch 243
Validation auc = 0.661502
Epoch 244
Loss = 1.6525e-01, PNorm = 24.4641, GNorm = 1.7414, lr_0 = 6.6625e-04
Validation auc = 0.668545
Epoch 245
Validation auc = 0.671831
Epoch 246
Validation auc = 0.677934
Epoch 247
Validation auc = 0.681690
Epoch 248
Validation auc = 0.679343
Epoch 249
Loss = 2.0294e-01, PNorm = 24.5382, GNorm = 3.7347, lr_0 = 6.6056e-04
Validation auc = 0.681690
Epoch 250
Validation auc = 0.686385
Epoch 251
Validation auc = 0.685446
Epoch 252
Validation auc = 0.684507
Epoch 253
Validation auc = 0.687324
Epoch 254
Loss = 1.3888e-01, PNorm = 24.6016, GNorm = 1.2387, lr_0 = 6.5491e-04
Validation auc = 0.685915
Epoch 255
Validation auc = 0.682629
Epoch 256
Validation auc = 0.681690
Epoch 257
Validation auc = 0.677465
Epoch 258
Validation auc = 0.676995
Epoch 259
Loss = 1.7197e-01, PNorm = 24.6635, GNorm = 2.0071, lr_0 = 6.4931e-04
Validation auc = 0.676526
Epoch 260
Validation auc = 0.676995
Epoch 261
Validation auc = 0.677934
Epoch 262
Validation auc = 0.678873
Epoch 263
Validation auc = 0.680751
Epoch 264
Loss = 1.5590e-01, PNorm = 24.7160, GNorm = 1.9257, lr_0 = 6.4376e-04
Validation auc = 0.679812
Epoch 265
Validation auc = 0.681221
Epoch 266
Validation auc = 0.680282
Epoch 267
Validation auc = 0.678404
Epoch 268
Validation auc = 0.676056
Epoch 269
Loss = 1.4000e-01, PNorm = 24.7765, GNorm = 1.7758, lr_0 = 6.3826e-04
Validation auc = 0.676995
Epoch 270
Validation auc = 0.670423
Epoch 271
Validation auc = 0.670892
Epoch 272
Validation auc = 0.665258
Epoch 273
Validation auc = 0.657746
Epoch 274
Loss = 9.4901e-02, PNorm = 24.8390, GNorm = 1.6591, lr_0 = 6.3280e-04
Validation auc = 0.655399
Epoch 275
Validation auc = 0.662441
Epoch 276
Validation auc = 0.670892
Epoch 277
Validation auc = 0.675587
Epoch 278
Validation auc = 0.674648
Epoch 279
Loss = 1.4440e-01, PNorm = 24.8868, GNorm = 2.1687, lr_0 = 6.2739e-04
Validation auc = 0.675587
Epoch 280
Validation auc = 0.677465
Epoch 281
Validation auc = 0.680751
Epoch 282
Validation auc = 0.679343
Epoch 283
Validation auc = 0.676995
Epoch 284
Loss = 1.6778e-01, PNorm = 24.9356, GNorm = 2.0499, lr_0 = 6.2203e-04
Validation auc = 0.677934
Epoch 285
Validation auc = 0.675117
Epoch 286
Validation auc = 0.667136
Epoch 287
Validation auc = 0.666667
Epoch 288
Validation auc = 0.667136
Epoch 289
Loss = 9.3411e-02, PNorm = 24.9876, GNorm = 2.3893, lr_0 = 6.1671e-04
Validation auc = 0.662911
Epoch 290
Validation auc = 0.655869
Epoch 291
Validation auc = 0.663850
Epoch 292
Validation auc = 0.666197
Epoch 293
Validation auc = 0.671362
Epoch 294
Loss = 2.3124e-01, PNorm = 25.0352, GNorm = 1.1695, lr_0 = 6.1144e-04
Validation auc = 0.673239
Epoch 295
Validation auc = 0.672300
Epoch 296
Validation auc = 0.672300
Epoch 297
Validation auc = 0.667136
Epoch 298
Validation auc = 0.665728
Epoch 299
Loss = 1.0664e-01, PNorm = 25.0788, GNorm = 0.8658, lr_0 = 6.0621e-04
Validation auc = 0.671831
Model 0 best validation auc = 0.749765 on epoch 35
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.862733
Ensemble test auc = 0.862733
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 241,601
Moving model to cuda
Epoch 0
Validation auc = 0.500469
Epoch 1
Validation auc = 0.571362
Epoch 2
Validation auc = 0.612676
Epoch 3
Validation auc = 0.654460
Epoch 4
Loss = 7.1326e-01, PNorm = 20.9848, GNorm = 2.5132, lr_0 = 6.5000e-04
Validation auc = 0.646009
Epoch 5
Validation auc = 0.627230
Epoch 6
Validation auc = 0.598122
Epoch 7
Validation auc = 0.606573
Epoch 8
Validation auc = 0.620188
Epoch 9
Loss = 5.1861e-01, PNorm = 21.0313, GNorm = 6.0662, lr_0 = 9.9743e-04
Validation auc = 0.581221
Epoch 10
Validation auc = 0.602817
Epoch 11
Validation auc = 0.622066
Epoch 12
Validation auc = 0.586385
Epoch 13
Validation auc = 0.581221
Epoch 14
Loss = 4.4190e-01, PNorm = 21.0887, GNorm = 2.5657, lr_0 = 9.8890e-04
Validation auc = 0.613615
Epoch 15
Validation auc = 0.635681
Epoch 16
Validation auc = 0.632864
Epoch 17
Validation auc = 0.635211
Epoch 18
Validation auc = 0.669484
Epoch 19
Loss = 4.2002e-01, PNorm = 21.1480, GNorm = 1.9903, lr_0 = 9.8045e-04
Validation auc = 0.665258
Epoch 20
Validation auc = 0.701878
Epoch 21
Validation auc = 0.715023
Epoch 22
Validation auc = 0.722535
Epoch 23
Validation auc = 0.730047
Epoch 24
Loss = 3.8708e-01, PNorm = 21.2182, GNorm = 1.7939, lr_0 = 9.7207e-04
Validation auc = 0.737089
Epoch 25
Validation auc = 0.737089
Epoch 26
Validation auc = 0.722066
Epoch 27
Validation auc = 0.708451
Epoch 28
Validation auc = 0.703286
Epoch 29
Loss = 3.3344e-01, PNorm = 21.2970, GNorm = 3.7063, lr_0 = 9.6376e-04
Validation auc = 0.700000
Epoch 30
Validation auc = 0.713146
Epoch 31
Validation auc = 0.705164
Epoch 32
Validation auc = 0.701408
Epoch 33
Validation auc = 0.708920
Epoch 34
Loss = 3.3485e-01, PNorm = 21.3820, GNorm = 1.8909, lr_0 = 9.5552e-04
Validation auc = 0.716432
Epoch 35
Validation auc = 0.717840
Epoch 36
Validation auc = 0.714085
Epoch 37
Validation auc = 0.710329
Epoch 38
Validation auc = 0.708920
Epoch 39
Loss = 4.3759e-01, PNorm = 21.4598, GNorm = 6.6565, lr_0 = 9.4735e-04
Validation auc = 0.703756
Epoch 40
Validation auc = 0.703756
Epoch 41
Validation auc = 0.704225
Epoch 42
Validation auc = 0.704695
Epoch 43
Validation auc = 0.704225
Epoch 44
Loss = 4.3637e-01, PNorm = 21.5394, GNorm = 2.4223, lr_0 = 9.3925e-04
Validation auc = 0.712207
Epoch 45
Validation auc = 0.711268
Epoch 46
Validation auc = 0.707512
Epoch 47
Validation auc = 0.715962
Epoch 48
Validation auc = 0.711737
Epoch 49
Loss = 3.2315e-01, PNorm = 21.6184, GNorm = 1.6050, lr_0 = 9.3122e-04
Validation auc = 0.707512
Epoch 50
Validation auc = 0.700000
Epoch 51
Validation auc = 0.696714
Epoch 52
Validation auc = 0.695305
Epoch 53
Validation auc = 0.679812
Epoch 54
Loss = 2.6966e-01, PNorm = 21.7014, GNorm = 1.8723, lr_0 = 9.2326e-04
Validation auc = 0.669953
Epoch 55
Validation auc = 0.690141
Epoch 56
Validation auc = 0.692488
Epoch 57
Validation auc = 0.690141
Epoch 58
Validation auc = 0.689202
Epoch 59
Loss = 3.0276e-01, PNorm = 21.7851, GNorm = 4.3891, lr_0 = 9.1537e-04
Validation auc = 0.685446
Epoch 60
Validation auc = 0.682629
Epoch 61
Validation auc = 0.677465
Epoch 62
Validation auc = 0.683568
Epoch 63
Validation auc = 0.686385
Epoch 64
Loss = 3.0862e-01, PNorm = 21.8811, GNorm = 2.3046, lr_0 = 9.0754e-04
Validation auc = 0.688263
Epoch 65
Validation auc = 0.680282
Epoch 66
Validation auc = 0.663380
Epoch 67
Validation auc = 0.635681
Epoch 68
Validation auc = 0.628638
Epoch 69
Loss = 2.4474e-01, PNorm = 21.9785, GNorm = 2.1748, lr_0 = 8.9978e-04
Validation auc = 0.623005
Epoch 70
Validation auc = 0.616901
Epoch 71
Validation auc = 0.623944
Epoch 72
Validation auc = 0.641784
Epoch 73
Validation auc = 0.680751
Epoch 74
Loss = 2.5578e-01, PNorm = 22.0824, GNorm = 1.7478, lr_0 = 8.9209e-04
Validation auc = 0.685915
Epoch 75
Validation auc = 0.683568
Epoch 76
Validation auc = 0.685446
Epoch 77
Validation auc = 0.679812
Epoch 78
Validation auc = 0.686854
Epoch 79
Loss = 3.1845e-01, PNorm = 22.1788, GNorm = 2.6213, lr_0 = 8.8447e-04
Validation auc = 0.682160
Epoch 80
Validation auc = 0.692958
Epoch 81
Validation auc = 0.685446
Epoch 82
Validation auc = 0.686385
Epoch 83
Validation auc = 0.681221
Epoch 84
Loss = 3.0025e-01, PNorm = 22.2771, GNorm = 3.4931, lr_0 = 8.7691e-04
Validation auc = 0.681690
Epoch 85
Validation auc = 0.684977
Epoch 86
Validation auc = 0.684038
Epoch 87
Validation auc = 0.690141
Epoch 88
Validation auc = 0.689671
Epoch 89
Loss = 2.7728e-01, PNorm = 22.3629, GNorm = 2.4528, lr_0 = 8.6941e-04
Validation auc = 0.693897
Epoch 90
Validation auc = 0.691080
Epoch 91
Validation auc = 0.688732
Epoch 92
Validation auc = 0.686854
Epoch 93
Validation auc = 0.685915
Epoch 94
Loss = 2.8883e-01, PNorm = 22.4428, GNorm = 3.1986, lr_0 = 8.6198e-04
Validation auc = 0.679812
Epoch 95
Validation auc = 0.674648
Epoch 96
Validation auc = 0.677934
Epoch 97
Validation auc = 0.674178
Epoch 98
Validation auc = 0.682160
Epoch 99
Loss = 2.6347e-01, PNorm = 22.5333, GNorm = 1.6422, lr_0 = 8.5461e-04
Validation auc = 0.684507
Epoch 100
Validation auc = 0.681221
Epoch 101
Validation auc = 0.676526
Epoch 102
Validation auc = 0.692488
Epoch 103
Validation auc = 0.692019
Epoch 104
Loss = 2.6695e-01, PNorm = 22.6342, GNorm = 4.4115, lr_0 = 8.4730e-04
Validation auc = 0.688732
Epoch 105
Validation auc = 0.689671
Epoch 106
Validation auc = 0.686854
Epoch 107
Validation auc = 0.686854
Epoch 108
Validation auc = 0.692019
Epoch 109
Loss = 1.7421e-01, PNorm = 22.7170, GNorm = 1.7052, lr_0 = 8.4006e-04
Validation auc = 0.699061
Epoch 110
Validation auc = 0.701878
Epoch 111
Validation auc = 0.695775
Epoch 112
Validation auc = 0.692019
Epoch 113
Validation auc = 0.690610
Epoch 114
Loss = 2.2146e-01, PNorm = 22.8000, GNorm = 1.6080, lr_0 = 8.3288e-04
Validation auc = 0.683568
Epoch 115
Validation auc = 0.674178
Epoch 116
Validation auc = 0.661502
Epoch 117
Validation auc = 0.634742
Epoch 118
Validation auc = 0.659155
Epoch 119
Loss = 2.0707e-01, PNorm = 22.8741, GNorm = 3.4389, lr_0 = 8.2576e-04
Validation auc = 0.677465
Epoch 120
Validation auc = 0.678873
Epoch 121
Validation auc = 0.679343
Epoch 122
Validation auc = 0.686854
Epoch 123
Validation auc = 0.691080
Epoch 124
Loss = 2.0053e-01, PNorm = 22.9516, GNorm = 1.6024, lr_0 = 8.1870e-04
Validation auc = 0.691080
Epoch 125
Validation auc = 0.692019
Epoch 126
Validation auc = 0.688263
Epoch 127
Validation auc = 0.678404
Epoch 128
Validation auc = 0.671831
Epoch 129
Loss = 1.4998e-01, PNorm = 23.0367, GNorm = 2.5432, lr_0 = 8.1170e-04
Validation auc = 0.667136
Epoch 130
Validation auc = 0.665258
Epoch 131
Validation auc = 0.659624
Epoch 132
Validation auc = 0.650704
Epoch 133
Validation auc = 0.651174
Epoch 134
Loss = 1.8182e-01, PNorm = 23.1004, GNorm = 1.7863, lr_0 = 8.0476e-04
Validation auc = 0.658216
Epoch 135
Validation auc = 0.669953
Epoch 136
Validation auc = 0.669484
Epoch 137
Validation auc = 0.665258
Epoch 138
Validation auc = 0.663850
Epoch 139
Loss = 1.9171e-01, PNorm = 23.1658, GNorm = 1.7084, lr_0 = 7.9788e-04
Validation auc = 0.661033
Epoch 140
Validation auc = 0.656808
Epoch 141
Validation auc = 0.654460
Epoch 142
Validation auc = 0.656338
Epoch 143
Validation auc = 0.669014
Epoch 144
Loss = 1.5480e-01, PNorm = 23.2365, GNorm = 1.3317, lr_0 = 7.9106e-04
Validation auc = 0.678873
Epoch 145
Validation auc = 0.674648
Epoch 146
Validation auc = 0.671831
Epoch 147
Validation auc = 0.677934
Epoch 148
Validation auc = 0.680282
Epoch 149
Loss = 1.9431e-01, PNorm = 23.3071, GNorm = 4.0325, lr_0 = 7.8430e-04
Validation auc = 0.685446
Epoch 150
Validation auc = 0.675587
Epoch 151
Validation auc = 0.681690
Epoch 152
Validation auc = 0.687793
Epoch 153
Validation auc = 0.670423
Epoch 154
Loss = 2.7642e-01, PNorm = 23.3795, GNorm = 2.8777, lr_0 = 7.7759e-04
Validation auc = 0.674648
Epoch 155
Validation auc = 0.677465
Epoch 156
Validation auc = 0.670423
Epoch 157
Validation auc = 0.650704
Epoch 158
Validation auc = 0.654460
Epoch 159
Loss = 1.5758e-01, PNorm = 23.4611, GNorm = 1.4603, lr_0 = 7.7095e-04
Validation auc = 0.653991
Epoch 160
Validation auc = 0.642254
Epoch 161
Validation auc = 0.617840
Epoch 162
Validation auc = 0.615493
Epoch 163
Validation auc = 0.615962
Epoch 164
Loss = 1.7488e-01, PNorm = 23.5341, GNorm = 1.3989, lr_0 = 7.6436e-04
Validation auc = 0.626291
Epoch 165
Validation auc = 0.621596
Epoch 166
Validation auc = 0.661033
Epoch 167
Validation auc = 0.676995
Epoch 168
Validation auc = 0.683568
Epoch 169
Loss = 1.5305e-01, PNorm = 23.6154, GNorm = 1.4990, lr_0 = 7.5782e-04
Validation auc = 0.684977
Epoch 170
Validation auc = 0.686385
Epoch 171
Validation auc = 0.692019
Epoch 172
Validation auc = 0.686854
Epoch 173
Validation auc = 0.685446
Epoch 174
Loss = 2.2551e-01, PNorm = 23.6773, GNorm = 1.6539, lr_0 = 7.5134e-04
Validation auc = 0.684038
Epoch 175
Validation auc = 0.680751
Epoch 176
Validation auc = 0.681221
Epoch 177
Validation auc = 0.681690
Epoch 178
Validation auc = 0.686854
Epoch 179
Loss = 2.6980e-01, PNorm = 23.7351, GNorm = 4.2803, lr_0 = 7.4492e-04
Validation auc = 0.686854
Epoch 180
Validation auc = 0.685446
Epoch 181
Validation auc = 0.683099
Epoch 182
Validation auc = 0.678404
Epoch 183
Validation auc = 0.673239
Epoch 184
Loss = 2.2323e-01, PNorm = 23.7906, GNorm = 2.5755, lr_0 = 7.3855e-04
Validation auc = 0.667606
Epoch 185
Validation auc = 0.675587
Epoch 186
Validation auc = 0.676056
Epoch 187
Validation auc = 0.676995
Epoch 188
Validation auc = 0.673709
Epoch 189
Loss = 1.4194e-01, PNorm = 23.8489, GNorm = 2.4125, lr_0 = 7.3224e-04
Validation auc = 0.669953
Epoch 190
Validation auc = 0.666667
Epoch 191
Validation auc = 0.664319
Epoch 192
Validation auc = 0.663380
Epoch 193
Validation auc = 0.667606
Epoch 194
Loss = 1.6228e-01, PNorm = 23.9201, GNorm = 2.5650, lr_0 = 7.2598e-04
Validation auc = 0.665258
Epoch 195
Validation auc = 0.669953
Epoch 196
Validation auc = 0.673239
Epoch 197
Validation auc = 0.676526
Epoch 198
Validation auc = 0.676056
Epoch 199
Loss = 2.2084e-01, PNorm = 23.9890, GNorm = 2.4425, lr_0 = 7.1977e-04
Validation auc = 0.672300
Epoch 200
Validation auc = 0.654460
Epoch 201
Validation auc = 0.656808
Epoch 202
Validation auc = 0.669953
Epoch 203
Validation auc = 0.675587
Epoch 204
Loss = 2.4567e-01, PNorm = 24.0593, GNorm = 1.8609, lr_0 = 7.1362e-04
Validation auc = 0.670423
Epoch 205
Validation auc = 0.675587
Epoch 206
Validation auc = 0.673239
Epoch 207
Validation auc = 0.667606
Epoch 208
Validation auc = 0.653052
Epoch 209
Loss = 1.6255e-01, PNorm = 24.1247, GNorm = 1.7871, lr_0 = 7.0752e-04
Validation auc = 0.646009
Epoch 210
Validation auc = 0.655869
Epoch 211
Validation auc = 0.676056
Epoch 212
Validation auc = 0.679812
Epoch 213
Validation auc = 0.677465
Epoch 214
Loss = 1.6369e-01, PNorm = 24.1933, GNorm = 2.3252, lr_0 = 7.0147e-04
Validation auc = 0.676056
Epoch 215
Validation auc = 0.682160
Epoch 216
Validation auc = 0.690141
Epoch 217
Validation auc = 0.690610
Epoch 218
Validation auc = 0.678404
Epoch 219
Loss = 1.6225e-01, PNorm = 24.2639, GNorm = 4.6566, lr_0 = 6.9548e-04
Validation auc = 0.669014
Epoch 220
Validation auc = 0.676995
Epoch 221
Validation auc = 0.695775
Epoch 222
Validation auc = 0.696714
Epoch 223
Validation auc = 0.691080
Epoch 224
Loss = 1.9759e-01, PNorm = 24.3330, GNorm = 0.9651, lr_0 = 6.8953e-04
Validation auc = 0.689202
Epoch 225
Validation auc = 0.687324
Epoch 226
Validation auc = 0.687793
Epoch 227
Validation auc = 0.685915
Epoch 228
Validation auc = 0.671831
Epoch 229
Loss = 1.4981e-01, PNorm = 24.4004, GNorm = 2.5822, lr_0 = 6.8364e-04
Validation auc = 0.668075
Epoch 230
Validation auc = 0.665728
Epoch 231
Validation auc = 0.670892
Epoch 232
Validation auc = 0.681221
Epoch 233
Validation auc = 0.678873
Epoch 234
Loss = 1.4311e-01, PNorm = 24.4609, GNorm = 2.5643, lr_0 = 6.7779e-04
Validation auc = 0.675117
Epoch 235
Validation auc = 0.673239
Epoch 236
Validation auc = 0.672770
Epoch 237
Validation auc = 0.670423
Epoch 238
Validation auc = 0.669953
Epoch 239
Loss = 1.9330e-01, PNorm = 24.5217, GNorm = 2.4417, lr_0 = 6.7200e-04
Validation auc = 0.663850
Epoch 240
Validation auc = 0.656338
Epoch 241
Validation auc = 0.653521
Epoch 242
Validation auc = 0.662441
Epoch 243
Validation auc = 0.671362
Epoch 244
Loss = 1.2795e-01, PNorm = 24.5693, GNorm = 1.3925, lr_0 = 6.6625e-04
Validation auc = 0.673709
Epoch 245
Validation auc = 0.673239
Epoch 246
Validation auc = 0.671831
Epoch 247
Validation auc = 0.669014
Epoch 248
Validation auc = 0.660094
Epoch 249
Loss = 1.9250e-01, PNorm = 24.6250, GNorm = 1.7643, lr_0 = 6.6056e-04
Validation auc = 0.654930
Epoch 250
Validation auc = 0.651174
Epoch 251
Validation auc = 0.631925
Epoch 252
Validation auc = 0.642723
Epoch 253
Validation auc = 0.659155
Epoch 254
Loss = 2.5987e-01, PNorm = 24.6892, GNorm = 5.5143, lr_0 = 6.5491e-04
Validation auc = 0.667606
Epoch 255
Validation auc = 0.663380
Epoch 256
Validation auc = 0.654460
Epoch 257
Validation auc = 0.653991
Epoch 258
Validation auc = 0.661502
Epoch 259
Loss = 1.7140e-01, PNorm = 24.7594, GNorm = 4.3322, lr_0 = 6.4931e-04
Validation auc = 0.664789
Epoch 260
Validation auc = 0.656808
Epoch 261
Validation auc = 0.647887
Epoch 262
Validation auc = 0.653991
Epoch 263
Validation auc = 0.660094
Epoch 264
Loss = 1.5509e-01, PNorm = 24.8196, GNorm = 2.1223, lr_0 = 6.4376e-04
Validation auc = 0.666667
Epoch 265
Validation auc = 0.671831
Epoch 266
Validation auc = 0.678404
Epoch 267
Validation auc = 0.685446
Epoch 268
Validation auc = 0.691549
Epoch 269
Loss = 1.8388e-01, PNorm = 24.8765, GNorm = 1.4827, lr_0 = 6.3826e-04
Validation auc = 0.693427
Epoch 270
Validation auc = 0.698592
Epoch 271
Validation auc = 0.694836
Epoch 272
Validation auc = 0.691080
Epoch 273
Validation auc = 0.686385
Epoch 274
Loss = 1.5302e-01, PNorm = 24.9303, GNorm = 1.6627, lr_0 = 6.3280e-04
Validation auc = 0.680282
Epoch 275
Validation auc = 0.671362
Epoch 276
Validation auc = 0.666197
Epoch 277
Validation auc = 0.661972
Epoch 278
Validation auc = 0.660094
Epoch 279
Loss = 1.8441e-01, PNorm = 24.9880, GNorm = 1.7551, lr_0 = 6.2739e-04
Validation auc = 0.659624
Epoch 280
Validation auc = 0.657746
Epoch 281
Validation auc = 0.659624
Epoch 282
Validation auc = 0.660563
Epoch 283
Validation auc = 0.659155
Epoch 284
Loss = 1.4993e-01, PNorm = 25.0433, GNorm = 2.0460, lr_0 = 6.2203e-04
Validation auc = 0.660094
Epoch 285
Validation auc = 0.662441
Epoch 286
Validation auc = 0.663380
Epoch 287
Validation auc = 0.668545
Epoch 288
Validation auc = 0.673239
Epoch 289
Loss = 8.6181e-02, PNorm = 25.1008, GNorm = 2.8700, lr_0 = 6.1671e-04
Validation auc = 0.674648
Epoch 290
Validation auc = 0.670892
Epoch 291
Validation auc = 0.670423
Epoch 292
Validation auc = 0.670423
Epoch 293
Validation auc = 0.681690
Epoch 294
Loss = 2.0018e-01, PNorm = 25.1632, GNorm = 3.7856, lr_0 = 6.1144e-04
Validation auc = 0.674178
Epoch 295
Validation auc = 0.669953
Epoch 296
Validation auc = 0.665258
Epoch 297
Validation auc = 0.675117
Epoch 298
Validation auc = 0.675587
Epoch 299
Loss = 1.3626e-01, PNorm = 25.2216, GNorm = 1.4663, lr_0 = 6.0621e-04
Validation auc = 0.674648
Model 0 best validation auc = 0.737089 on epoch 24
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.880795
Ensemble test auc = 0.880795
10-fold cross validation
	Seed 0 ==> test auc = 0.871764
	Seed 1 ==> test auc = 0.874774
	Seed 2 ==> test auc = 0.874774
	Seed 3 ==> test auc = 0.886213
	Seed 4 ==> test auc = 0.859121
	Seed 5 ==> test auc = 0.875376
	Seed 6 ==> test auc = 0.873570
	Seed 7 ==> test auc = 0.877784
	Seed 8 ==> test auc = 0.862733
	Seed 9 ==> test auc = 0.880795
Overall test auc = 0.873691 +/- 0.007519
Elapsed time = 0:03:05
